Everything You Need to Know About Avatars with Robby Ratan: Step into the Metaverse podcast: EP20

Good day everyone and welcome to the Step into the Metaverse podcast. My name is Marc Van Rimenop and I'm also known as the Digital Speaker. I'm a future tech strategist and I think about how emerging technologies change organizations and society. This podcast is part of my new book called Step into the how the Immersive Internet Will Unlock a Trillion Dollar Social Economy. The book is available as an ebook, hard copy and audiobook. As part of this book I did nearly 100 in depth interviews with the stakeholders involved in creating the open Metaverse, the investors, the developers, the artists, the collectors, etc. And I've turned a selection of these interviews into this podcast as I think that these conversations are too valuable to leave them on my computer. In this interview I interviewed Rabindra Ratan. Robby, as he is also known, is an Associate professor and AT&T Scholar at the Michigan State University's Department of Media and Information. His research focuses primarily on the psychological experience and effects of media use, with an emphasis on avatars, video games, virtual worlds, and other interactive environments. He is particularly interested in how different facets of mediated self representations, for example the self concept, influence the psychological experience of media use and how different facets of this psychological experience, e.g. embodiment or identification, affect a variety of outcomes, including education behaviors, e.g. classroom performance, health related behaviors, e.g. food choice, and prejudicial or pro social attitudes, e.g. gender or race bias. During the interview we discussed the role of avatars in depth, including what the requirements and characteristics of avatars will be. We discussed the risk of ending up in the Uncanny Valley as well as the risk of imposter avatars. Enjoy the conversation.

My first question really is if you could share a bit about your background and how you are involved with the Metaverse.

Sure, yeah. So I'm an Associate professor of Media and Information in the College of Communication Arts and Sciences at Michigan State University, and I've been studying what I would now describe as Metaverse related technologies since I was a PhD student. I finished in 2011 and I was at first inspired to do research on the Metaverse as a grad student in Jeremy Bailenson's lab at Stanford, where he was using virtual reality to examine how people interacted socially or with avatars in virtual worlds, and he encouraged me to read Snow Crash. This is about a decade after it was written and I thought the term and the idea of the metaverse. Also I read Neuromancer by Gibson right around that time and so these ideas of this future Internet were so fascinating to me that I wanted to spend my career studying these technologies. And so here I am. I've spent the past decade and a half bringing people into the lab, having them play video games, use virtual worlds, use avatars, customize avatars, assign assigned avatars. Then I measure how those manipulations in those lab experiments influence their social and psychological kind of behaviors and attitudes.

Okay, well, that's very interesting and very relevant, but the first question I have, and this question I ask everyone, and that is, what is the Metaverse?

Sure. Most simply, the Metaverse is an interoperable network of virtual or immersive spaces that are navigable through a Metaverse browser, which is like a Web browser, which you can access through a VR headset in order to be, you know, fully immersed or through other technologies, AR. Even your phone to some extent. And it is the next level or layer of the Internet that is facilitated. That interoperability is facilitated by blockchain technologies which keep track of the flow of goods, such as avatars or NFTs. Your identity, your reputation, all of these things can move between virtual worlds. Unlike in the past, where if you buy an avatar in Fortnite, it's stuck in Fortnite. Well, you can travel through these spaces because there's this public record of who owns what, and that's what makes it a metaverse. The Metaverse is not a single virtual world alone. Any company saying we have our metaverse is lying, because that's not the idea. The idea is that it's like the World Wide Web. It's an interconnection of all these spaces. And fine, you can have your own company Internet, and it's not on the World Wide Web. It's not on the World Wide Web. So in order to be part of the Web, you have to be part of the network. Same idea with the Metaverse.

Yeah, that makes total sense to me as well. So if we dive into the world of avatars, avatars will be, you could say, some of the usernames and social profiles of the Metaverse. What are some of the main characteristics and requirements of an avatar in the Metaverse?

So the way I define an avatar, which is based on some very careful social scientific researcher thinking, so it's not necessarily the colloquial response, but we define it as a representation of an intelligent entity that facilitates interactions with other people, objects, or the environment. Okay, so the key components here are that it's a representation. It can be a username, like you say, but it can also be an audio signal. It doesn't matter what medium the representation is coming through. It's a representation of an intelligent entity. So by definition, we do not use, use the term avatars to refer to agents or bots, you know, computer programs, unless they're intelligent. Right. So if you know, HAL 2000 becomes self aware and has an avatar that represents HAL, then I would argue, fine, it's an avatar because it's a representation, but it's not the person themselves. And it's not, you know, some dumb bot, some NPC that's unintelligent. And then it has to facilitate interactions. That's the third piece of it. So a picture of yourself on the wall, not an avatar. Even if it's a digital picture in your virtual bedroom, still not an avatar unless you're using it to interact with people. And that's where the definition gets a little fuzzy. Because how synchronous does that interaction need to be in order for us to consider it an avatar or not? Well, an avatar is in the eye of the beholder on that one. But the main characteristics then are it allows you to interact with, with other people. So it could be like a social profile, like you said. And another characteristic that is implicit in this definition is that you have control over this representation. And that means control through synchronous real time interactions. You know, it moves when I want it to move, but also you control how it appears or how it sounds or how it feels. Maybe even how it smells and tastes someday. But those are kind of futuristic in a way. Ideas about a very fundamental aspect of avatars, which is you can be, to the extent allowed by the platform, what you want to be as opposed to what you're born with. And that creates huge possibilities in their uses and the outcomes of their uses.

Yeah, that makes total sense. So, and how important are NFTs when it comes to avatars? Not so much JPEGs being sold for millions, but when it comes to avatars, very important.

I think it's the bedrock of the future economy. You know, if you want to distinguish yourself from your peers in order to be cool and yet follow a trend, which is what all teenagers want to do, they buy. I got to have that cool T shirt with that cool logo, but I can't have the same one my friends have. That's the nft, right? That's the avatar. No one else has this avatar. You know, my rich daddy had it custom made for me, or, you know, I worked really hard at Burger King and bought myself an avatar. And so the ability to own and take this digital object with you through virtual space to represent you is very important. I would actually argue that it's probably the most personally important economic aspect of the Metaverse.

Wow. I think that makes sense because Metaverse allows us to, as you said, to be whoever we want to be. And identity in the virtual world will probably be as important as in the real world. So are you aware of the platforms that merge the Avatars and NFTs, and what are some lessons learned from these platforms?

I don't think we're advanced enough to learn lessons yet. We are still in the extremely experimental Wild west stage, and adoption now is not going to reflect adoption in the future. It's just too many early adopters who are not reflective of the general population. So people are experimenting with ideas like this. I think the biggest lesson I've seen. You mentioned Ready Player Me in your question here. Ready Player Me has the right idea. They are putting all of their kind of, I guess, tactical approach to the platform toward interoperability and ease of access. Right. You can make that avatar with your phone, web browser without an account, using a camera, and within five minutes, you can be inside of it in a virtual world, in VR chat. And when I did that, that was a really amazing moment for me. I realized, wow, we've gotten here. This technology is final. Somebody's realized, oh, we just have to make it really easy for people to use. And so I'm very impressed with Ready Player Me. So the lesson learned there is that by having these interoperability standards in the same way as I can bring a JPEG from one email client to web browser, to PowerPoint software, if I can bring the same avatar between these spaces, that is an aspect of the Metaverse that kind of facilitates that sense of cohesion between the different worlds.

Yeah. And what other platforms do you see, like, apart from Ready Player Me, that.

Are involved in space, particularly on the avatar side or.

Yeah, yeah.

Well, Lumi AI was pretty good one, but they were acquired by Roblox, and I think they're sunsetting their product. But that was avatars that were easily integrated into Zoom. But another platform like that is animes. Yeah, great bet on the future of the Metaverse right there. They allow you to use this 3D avatar that can be tracked in real time. They'll take in a Ready Player Me avatar or use another one that is made on their platform. And I think they've got a cool product that is ready to be kind of made a little like that ease of access aspect of Ready Player Me, I think anime could take a note from them and get bigger. Any other good ones for avatar creation? I mean, all the big social VR platforms, right? VR chat, all space, even Mozilla Hubs, they do try to make avatar creation as easy as possible and personal as possible. But not really sure how I feel about places like Decentraland or Somnium Space, et cetera. Like they want to get in on the NFT crypto blockchain side of the metaverse, but I think they're missing the boat. I think there's a lot of hype around virtual land. That's not the metaverse. You don't need to buy web space first and then figure out what to do with your website. You need a good idea for your website before you invest in the space. And so I think the model is a little bit backwards there. I think the benefit of the metaverse is the immersion. The user experience is the immersion, that kind of social presence or spatial or self presence in the avatar you experience. And so if all these platforms are giving you avatars, but then you can only get there on your flat screen computer, like you're there missing the boat. Or at least they're just not there yet. Like they hopefully will get there soon. But I'm a little disappointed.

It's a good point. I agree with that. I think especially digital real estate for me is artificial scarcity in terms of land in the space. It's infinite and it doesn't make sense to me.

Yeah, it's a bit overpriced, but eventually, I mean, you need web space, you're going to need metaverse space too. The market just needs to kind of equalize or hit an equilibrium.

Yeah, I agree. So if we look at the types of avatars, you know, we have Horizon, legendless avatar, very cartoony. And then you have Ready Player Me, which are, you know, quite still cartoony, but at least they're full body. And then you have the digital humans, metahumans, which are hyper realistic avatars. What is the direction that we will go? What do you think would become the standard?

That's a great question. This is something my research team and I talk about a lot. It's hard to predict. On the one hand, there's a theory that a cartoonish avatar that behaves in ways that are realistic is the best. Because with realism cues, the more realistic it is, the more you expect it to be like a human. And if it's so realistic, as you very well know through these 76 other conversations, the uncanny valley will turn People right off. This is the idea. Do you want me to explain it or you've already.

Yeah, I'm already cool.

But there's also something which you might not have heard of, called the uncanny valley of the mind. Have you heard of that?

No, I haven't heard that.

So the uncanny valley is essentially a mismatch between your expectation in the visual characteristics of the avatar or digital character or whatever it is. The uncanny valley of the mind is essentially the same thing, but for behavior instead of appearance. But it takes appearance into account. If an avatar seems to imply humanness to any extent, you know, even just eyes, a nose, ears and a mouth on a refrigerator signifies social potential. But then if it doesn't behave in a way that is sufficiently human, we are immediately disappointed by it. I hypothesize this is why Siri and Google Assistant and Cortana Alexa, none of them have a body. If they have a body, then we would be so angry when they mess up what we're telling them to do. But if they're disembodied, we can forgive them a little more easily. And the same idea, I think can be applied to the way that these avatars are behaving in virtual spaces. I think Facebook made a good move and I haven't played around much in her in horizon spaces. But the cartoonish avatar is hard to disappoint and behavior can be psychological, but it can also be physical, right? So if the movements are clunky, if there's a bit of lag or some jumpiness, it's easier to forgive a cartoonish avatar than it is a hyper realistic avatar. So I would suspect that in contexts where the movement is complex or there are many different users, so then the potential for lag is much higher because, you know, the server has to manage displaying all the inputs to all the people at the same time and of course recording those inputs. So in those cases, cartoonish avatars will be better. When you and I are sitting here on a zoom call, a hyper realistic avatar will be, I think, better. You know, I'm not, I'm not shaking around very much. We're just looking right at each other and talking like we would at a cafe or at a desk in an office. In that case, I think hyper realistic is better. And because you don't end up, you know, breaking that expectation. And then also the more anthropomorphic the avatar is, you know, the more human like it is, the more people respond positively to it. So if you're doing a business deal, if you're you know, asking someone to marry you or getting married or you know, doing something serious. You're going to want those hyper realistic avatars, I believe, because they will be a better replacement of the so called real experience.

That's super interesting. Do you have any, any research papers on this topic that you could share?

The uncanny valley of the mind is a paper from Fox and Novak. Or it's Novak and Fox. All right, I'll make a note to send you their avatar definition paper. And then the realism stuff, I mean, their paper does anthropomorphism and realism. Their paper does a great job. I base a fair amount of my avatar psychology class on their work.

Okay, well that would be great because I think that's very important topic to discuss.

Yeah. And did you come across Jamie Banks's book, Avatars Assembled?

No, I didn't.

It's a good one. I mean, it's got chapters from different authors. It's not all, all by her. It's an edited volume. But some of the chapters are quite good on specific topics that you might find interesting. Okay, so how big is the threat of imposter avatars?

Well, I think this one is an interesting one because this is, I think, yeah, if you see what happens in Web2 with deepfakes, this is highly problematic. I could easily create an avatar of you and pretend to be you, even with your voice while not being you. How are we going to deal with this?

So certainly it's a problem to consider and psychologically it might be hard to accept that it's not you, even though it looks like you. But here's the irony. With a deep fake, it looks like a photorealistic version of yourself with an avatar. If it's. Maybe this is why the NFT market blows up. Because with an avatar as an nft, you can prove whether or not it's yours. Right. Like that's on the blockchain, that's in the ledger. So you could imagine technologies developing around that idea to make deep fakes with my avatar impossible. Because even though it might look the same to the human eye. Right. There's a digital code embedded right in there. You know, you put on your special AR goggles and you see the code, you know, you turn on the filter and it tells you whether or not it's owned by the person being claimed as the owner. So I actually see that as is a huge problem that is very solvable with this technology.

Ah, it's interesting, I guess, if you think about it, that makes sense. So, and then a Different question in terms of interoperability. You know, everyone talks about interoperability, but let's say that we managed to get interoperability. And then I have my Fortnite character with an axe, and I get into my business meeting room. I'm going to kill all my colleagues with my Fortnite weapon. That's not going to work. How to go about that?

Don't think about it as an interoperability of your avatar. Think about it as an interoperability of your closet, your avatar wardrobe, your wallet. Your crypto wallet carries all of your outfits, and they're all just different avatars. And it's easy as clicking a couple of buttons to switch from avatar A to avatar B. That's socially appropriate, depending on who you're going to be with.

But then I can also take my assets, so my NFTs, and all of.

Your NFTs, all of your cryptocurrencies, and your driver's license. Right. Like, imagine what do you keep in your wallet now? So you also keep your identity card, and that's a very important aspect of your crypto wallet.

But then I can take my. My bazooka from Grand Theft Auto and blow up the meeting room. I'm just.

Oh, I see. Oh, okay. Cool. I got you. Yeah, I think that's a solvable problem. I mean, it depends on where your colleagues plan to meet. So there'll be different zones. Did you ever hear of the game called Eve Online?

Not sure.

It's a big space opera. Well, I don't know if it's opera, but it's an MMO like World of Warcraft, but online, and all about spreadsheets. And I just associate it with very mathematically oriented people. It's a hard game to play. Well, I think about that game because. And in like many MMOs, there's low security space and high security space. And if you attack another player in high security space, you'll get blown up by the game watchers. But in low security space, it is possible, you can imagine, just like on different websites where there are rules of engagement. Or, like, think about it as Facebook groups. And this Facebook group, you are allowed to post links to other websites. In this other Facebook group, you're not allowed to. Or there are bots looking for curse words. Right. And if you. You are allowed to use curse words in one or another. Well, these platforms will essentially have the same settings. It's just a setting. Are you allowed to bring weapons in this space? Yes or no? Do weapons work? If they click yes, then sure, go blow up your colleagues because they were dumb enough to set the room in that meeting.

Okay, fair enough. That makes sense. I guess if you look at from that perspective, you know, we do need the standards to enable that so that yes, you can have the rules, but then they need to be able to work with particular standards that define the weapons as well.

Exactly. And I think that's why some platforms are going to win over others. Right. Like Facebook's not going to own the whole Metaverse, but Facebook is way ahead of everyone in terms of thinking about office appropriate spaces. So you may come to trust Meadows offices more than you might trust. I don't know, Firefox's offices, though, actually they're pretty far ahead of the game. I don't know who I would say isn't there yet. But you know, if they just aren't very good at making the settings easy to navigate, to make sure people aren't blowing each other up or toxic to each other or, you know, not maintaining personal space or things about usability like managing audio flows is a big issue in the Metaverse and virtual worlds. Have you ever spent time in a virtual world with multiple people trying to talk?

Yeah, it's very confusing.

It's so confusing. It's the worst. So, yeah, these are all things that aspects like settings of the space that really matter.

Yeah, that's a good point. And then when we look at the retail industry direct to Avatar, probably a new business model, how will that change the retail industry or the fashion industry this regard?

I think it's going to make these industries much richer in two regards. Literally, they're going to make a ton of money and then also richer in the sense that they won't just be selling you physical products, they'll be selling you linked products that have an NFT or a fungible token associated, you know, like a Bitcoin or a Gucci coin associated with your purse. And then you own both of those things when you buy the physical object and they both have value and you can both resell them or resell both of them. But when I go to resell my Gucci bag, it might not retain much value, but that coin that I bought, maybe that retains much more value. And so there's an added benefit to buying from them, which means they can charge more in good old paper money and although of course it's all done on credit card and it's digital anyway, but you know, in whatever nationally backed currency. So I think these industries that are already using 3D modeling and kind of advanced methodologies for digitally enhanced design of their physical products. They're pretty much prepared to start selling the digital product as well. And then people start just selling the digital product. Right. So some kid who learns how to 3D model, they enter the industry, and that's a whole other economic platform. And there's this great argument. So do you know Jeremy Bailenson? Have you talked to him or reached out to him?

I'm not sure.

So his book is called Experience on Demand, his recent book, and he opens it with the anecdote of how in 2014, Mark Zuckerberg goes to his lab at Stanford and tries out demos, all the VR stuff, and then, like, a month later buys Oculus for whatever.

Oh, really?

Yeah. Two billion.

That's interesting. Yeah.

And I'm reading his book right now because I'm. I'm teaching a class based on his book. And he's a friend. He's my former mentor, and he makes a really interesting argument that not only will the metaverse be good for the environment because it will reduce travel, but also so much conspicuous consumption happens in the Western world. You know, we buy all sorts of shit we don't need to show our neighbors that we have the shit that they don't have that they don't need either. But if now these are digital assets instead of physical assets, maybe we'll waste a little bit less. Maybe we'll fill landfills a little bit slower. I mean, it might be an optimistic argument, but nonetheless, it's one that I think we should promote.

Yeah, that's a very important point. Again, I think sustainability is important. So my final question, in terms of sort of the ethical challenges and the danger, you know, what do you foresee for the Metaverse from an avatar perspective?

Your avatar needs to track your movements, and aspects of your identity are embedded in those movements in order for it to operate properly. And that data is potentially medically sensitive, such as, you know, if you have a tremor, that kind of shows you might end up with Parkinson's. Or if you turn your heads towards certain objects and have attention, focus on certain objects that might reflect attraction or sexual orientation. There are many aspects of this data. Right now, I know what I'm clicking on. I mean, certainly the websites know, in a way, what you're attracted to, what types of medical information you might be looking up online. But it's not as reliable a source of information about you as these data will be. They'll definitely know exactly how tall you are, aspects of your kind of fitness, especially if there are Heart rate or skin conductance trackers. So I think that's a huge danger in privacy. And so we need laws like HIPAA in the United States to, or at least regulators to consider applying these types of laws to these types of data.

Very good points, because that is a danger. I think meta is already capable of eye tracking where you're looking, how long you're looking, the emotionally looked at, but also how your hands look like, how the room looks like, the people and the products in the room. It's all data being collected. That's quite challenging, I agree, but the movement from the avatar, didn't think about that yet. But that of course, you know, if you have a suit on that that monitors and if you shake, then that definitely can reflect something. That's a very good point.

Yeah. And it also identifies you, right? Like it's just as unique as a fingerprint, the way your body moves. And so even if you're quasi anonymous in wearing the headset, your body movements are not and yeah, wow, that's a good point. I'm really interested in the business models of the metaverse and the implications of those. Right now I'm working on an op ed about this idea and in traditional media we see that advertising often incentivizes kind of a base level of content, trying to appeal to the most number of people in order to fund the advertiser demand for the audience. And if the metaverse goes this route, then I think we're in trouble. But we have the potential to focus more on direct sales on commission, in the same way as you, you know, buy an app from the App Store. Apple makes money off of the App Store, Google makes money off of the Play Store. So if platforms focus more on those business models and less on advertising, perhaps we will end up with less antisocial content. Another aspect of the metaverse that I think is really important. And then I'll let you go. You must be so tired of these. I don't envy you.

Super interesting. I love it.

Do you know the Proteus effect?

No, I don't.

P R O T E U S It's this idea. I wrote a meta analysis of the papers that were. Right now there's over 50 papers published. But the meta analysis was a year ago, year and a half. So it's 46 at that time where basically if you use an avatar with certain characteristics, people conform to that avatar. So a taller avatar, you'll negotiate more aggressively in a kind of an economic negotiation. My favorite one is if you use an inventor Avatar, you know, with a lab coat and a clipboard. People in that condition, compared to street clothes, will come up with more creative ideas during a brainstorming task. So, yeah, it's pretty cool. It's been replicated like for exercise a healthy weight avatar people exercise more. The effect size is about R 0.2324. It's a little bit stronger for the studies in VR. That's what we found in the meta analysis and then in the follow up that we're running right now. So it's a cool finding, but it's a little bit unexpected and people don't realize like the avatar you use will influence how you behave. And this is a potential mechanism for influence that people might not be aware of.

Wow, that's super interesting. It's interesting because I've spoken to a lot of people and I think you're the first one of the speaker that has really gone deep dive into the avatars and a lot of people look at from the various different perspectives, but you have really focused on that and that's very interesting. And I think there's a lot more to the avatar than we are currently aware of. I think just stating the examples you just mentioned.

Cool. Yeah, yeah. There's two other talking points. Toxicity in gaming is a vicious cycle. But with avatars that allow you to mask gender, then potentially we could break the cycle of stereotypes. And the argument there is, you know, women and girls are dissuaded from these spaces because men are assholes to them as soon as they speak. And, you know, people realize that they're a woman. And so because they are dissuaded, then they participate less, which means they don't develop as much time and skill. But actually, in our studies, we've done studies with hundreds of thousands of players, women are just as good as men when you control for the amount of time they play. So, and this is across multiple games, multiple types of games. So potentially, if avatars can reduce the gender emphasis and instead focus on other aspects of identity, or mask your gender and your voice in a way that's reliable, then we can reduce toxicity and the harmful outcomes of that. The other one is zoom fatigue. Do you know zoom fatigue? Yeah, yeah, we just published an article about it. But basically we found that it's caused in part by facial dissatisfaction. Looking at your own face is bad, and that's why it's worse for women and some people of color, Asians compared to whites. So avatars are a solution there. Right. If we're meeting in virtual reality and in the metaverse, right? We're not looking at our faces. We're not as self conscious about our presentation.

Yeah, that's a very good point. Wow. I guess you'll have some a lot of reading material for me to extend my recent burn. I really appreciate it. I'm a fast reader and I use AI as well to help me in my research because otherwise it's impossible to get through all this. But thanks so much. I really appreciate it.

Sure, absolutely. Thanks, Mark. I'll talk to you later.

Thanks very much.

Have a good day.

Bye.

And that wraps up the conversation with Robby Raton. If there's one thing that stands out for me from the conversation, that is how avatars have the ability to change what it means to be human. Although avatars are digital entities, they can have real life effects. This is fascinating, which is why in my new book, Step into the Metaverse, I dedicated a full chapter on the avatars and how they change our identity. If you enjoyed this conversation, don't forget to hit the like and subscribe button on your favorite podcast platform so you will be notified when any of the other interviews will become available. Thank you and see you in the metaverse.