Why Digital Twins are Essential to Building the Metaverse with Richard Kerris - Step into the Metaverse podcast: EP24

Good day everyone and welcome to the Step into the Metaverse podcast. My name is Marc Van Reimenam and I'm also known as the Digital Speaker. I'm a future tech strategist and I think about how emerging technologies change organizations and society. This podcast is part of my new book called Step into the how the Immersive Internet Will Unlock a Trillion Dollar Social Economy. The book is available as an ebook, hard copy and audiobook. As part of this book I did nearly 100 in depth interviews with the stakeholders involved in creating the open Metaverse, the investors, the developers, the artists, the collectors, etc. And I've turned a selection of these interviews into this podcast as I think that these conversations are too valuable to leave them on my computer. In this interview I interviewed Richard Karras. Richard is the Vice President, Omniverse platform development at Nvidia. Nvidia's Omniverse is an easily extendable platform for 3D design, collaboration and scalable multi GPU real time, true to reality simulation. Richard has worked at Apple, Amazon, AWS and was the CTO of Lucasfilms. Prior to all that he was at Pixar. Richard has an extensive background in 3D and visual effects. During the interview we discussed the importance of interoperability and transferability of assets across the Metaverse. We talked about digital Twins and how the Omniverse platform can help enterprise organizations and how important content will be for the Metaverse. Enjoy the conversation.

Maybe you can share a bit of your background on how you are involved with the Metaverse.

So here at Nvidia, I'm the Vice President of our Omniverse developer platform. Been with the company for three years. I have an extensive background in 3D and visual effects and was CTO at Lucasfilm. I also have an extensive background around developer relations. I worked for Steve at Apple when the big comeback days and helped build the ecosystem around OS10. And prior to that, many, many years ago I was at a company startup that was called Pixar. So that was pretty fun and I go back any further, goes black and white and then cut out cards and.

All that kind of stuff.

I started making rock videos in the very early 80s and that led me to computer graphics. That led me to, you know, this great path that I've been on. So it's been a lot of fun. And so what we're doing here at Nvidia, we believe in Metaverse in the context of a description of virtual worlds that are connected. We're building Omniverse to connect those virtual worlds. We Think of this next iteration or generation of the World Wide Web as what people are terming as Metaverse. So there's many different facets to the web as you know. But if we look at the early days of the web, when it was the Internet, like 30 some years ago, we had, you know, chat channels, telnets and stuff that you could be like, I was an early member of the well for people that remember the well. And you could send emails or post on forums. And that was really what the connected start of the Internet was. Then there became the dot com bubble where all of a sudden imagery and things could be displayed because the speeds were getting better and things like that. And that that initial thing had a bunch of problems. And so that's why there was this.com bubble. And then everything kind of went away for a bit. But it wasn't because the Internet was wrong, and it wasn't because their thoughts on running businesses connected in the Internet were wrong. It was because there was no consistent foundation. There's no consistent plumbing. So depending on what browser you were on, on what machine you were on, what components you had installed, oh, you need Flash or you need Shockwave, oh, you need that. And you always had different experiences to whatever website you would go to. So from a business standpoint, if you don't have a consistency for the experience of your customers or consistency on how currency is managed, it's doomed to fail. And that's exactly what that first bubble burst was. Then came the advent of HTML. Now with HTML, we had a common foundation with which all companies could work together to advance. So even though it was the mid late 90s that it started, you had a common way of displaying text and a common way of displaying images. And companies worked together to ratify new standards until we had video and all that by in the 2000s. But by then, any website you went to, no matter what platform you were on, no matter what device you were on, you always had a consistent experience on the website that all of a sudden now had a common foundation for which businesses could build their livelihood on businesses such as commerce for consumers or enterprise businesses that would build a complete business based on this common foundation and the shared collective environment of what the web holds today. Five of the top 10 companies in the world are based on the World Wide Web. Thirty years ago, that was like a pipe dream. The reason why there's so much excitement on Metaverse is that we're at this similar point where people see if we have this common foundation, then all of these virtual worlds will Have a consistent way of being displayed, experienced, connected, and then it becomes the next generation of the World Wide Web. So that's kind of our top down view of why we think, you know, this is very exciting and why we are working very hard to build that foundation for people to build upon.

Yeah, that's just really, really good. And I think it shows that the standards are so important, as you say, to make this happen in the correct way. So you already mentioned, you know, that the metaverse is connected worlds, connected virtual worlds that are connected. Can you share a bit more about what the metaverse is, according to you? What does it entail? What are some of the characteristics?

Well, I think it's a description. I think it's a description like the universe is a description of planets. The metaverse can have many different types of virtual worlds. Virtual worlds that are game based that we see today. They're very active, they're the ones that are out there, but those are different than the ones that are connected to maybe a digital twin of a factory, where a company is trying to build a digital twin to understand how that factory will behave, perform, and the components in it adhere to the right characteristics. That's a different use case. But yet they could be connected. And why would you want to connect them? It's not that they need to be connected amongst themselves. It's so that the toolmakers and the people providing content are building once and deploying however they need to. If I make a video that's going to go on the web, doesn't matter what the content of the video is, because the content is based on the standard that we all ratified. So it'll play video no matter what kind of a website I'm on. And what excites us about this connectivity of virtual worlds is we think, as every company has a worldwide website today, small shop owners, musicians, large enterprises, all alike, the same will hold true with virtual worlds in a very, very near term. You know, you and I might have our own virtual worlds with which we've gone out and populated it with buildings and entertainment type things or projects we're working on, or studios that we might be involved with, galleries if we like art, and invite other people in and out of that, much the same way you do your home, your apartment or whatnot. Take it up a level. Now, businesses are going to have that experience. When, you know, for example, if you're going to go on a vacation, you might say, I want to go to this Hyatt out on, you know, Bora Bora. I wonder what it's like teleport into our virtual world and we'll walk you around and show you, we'll have a digital avatar that will welcome you. We'll have a complete true to reality simulation of our environment. You can walk around, you can pick out the room, you want all those things from the comfort of your own home before you've actually committed to it in the physical world. From a business standpoint, the idea of a factory that can be built in a virtual world in the Metaverse means that they can be saving, having cost savings of upwards of more than 30%. And we know that because we've been validating that with a project we've been working on with BMW and there's many others that are duplicating this idea. Where in that scenario, if you're producing millions of cars per year, every day matters, if a car's not coming out of that factory that day, your profit drops. You know all of that. So when you transition from one car to another in that factory line, you better make sure you don't skip a beat. But the reality is nowadays it sometimes takes weeks, if not months to make those decisions. You have heavy equipment, you sometimes have a lot of robotics that have to be retrained in the new layout. You have your employees that are going to have to deal with the way things are going to come and go and it might not be ergonomically safe for them and they may learn the hard way and there's factor and factor and factor. So by using a digital twin of that before you've committed to it in the physical world, you can have a gain, a lot of advantages there. For example, you can train the robots in the digital world and instead of just training one robot, you could train hundreds of them. So by the time they transfer that knowledge into the physical robot and in the physical world, it'll be many thousand times smarter. You can use digital humans that are AI driven to figure out what's the best way for them to move around the factory with the minimum amount of error prone type things that they could do, what's the best layout for the machinery. All of those things factor up more and more. But when you make those decisions many times in the digital world, but only make it once in the physical world, that's a huge cost saving. One last example I'll give you is, is a lot of times we're working with companies that build things like power stations and transportation stations and things like that. And there's two phases to the work that they Do. The first phase is designing and building that facility, and the second phase is the life of that facility. It becomes alive, it becomes monitored, people flow, heat flow, when to change things, when to make adjustments, et cetera. And it becomes a living, breathing thing. That living, breathing thing can exist in that virtual world, and they can monitor and manage it through that before they have to send a car or truck out to go change something on an antenna or go change this and that. So for them, digital Twins is an ongoing, living, breathing entity of how they manage their facilities. So that's just like. I've kind of like jumped around to a few things, but hopefully you could see the consistency in each of those things by having that common foundation. If you teleport from one site to another site in a virtual world, you want to make sure the 3D elements are drawn consistently, that your experience in that world with physics, if I drop something or if I hear something, et cetera, are done consistently. And that's the kind of commonality we looked at from analogy of the World Wide Web with HTML taking place. And that's why there's so much excitement about what, you know, is being termed as the metaverse. It's really the next generation of the web.

Yeah, I think this is a very useful, very interesting explanation and use cases of what the metaverse can bring us. So I'm also interested in the fact that everyone talks about the importance of interoperability, and I completely agree with that, because you want to use your identity or your avatar and you want to use all the various cases. I think it's the interoperability and the traversability so that you can easily move from one place to another. First part of the question is, how are we going to move? Because there is no map, there's no Google to find whatever you need. So how are we going to traverse the metaverse and then related to that, how are we going to make sure that it is interoperable, that my character from World of Warcraft is not going into a meeting room and killing all the colleagues, for example, because you brought a rocket, you know, from Grand Theft Auto. I don't know what. These are things that I think we need to have a good understanding of. What does the interoperability actually really mean?

Yeah, good question and great scenario. Hasn't there been a meeting where you wanted to bring the rocket launcher in some kind of.

Sure.

So let me start with the first part. How do you know what's out there and how will you navigate and learn? How have you learned what's out on the web, you know, it just becomes there's places to go to learn more about what's out on the web. You have Google search, you have, you know, you know, every business visit our website, you know, you have things that become common to the, you know, habit of what I'm looking for, including social gatherings of, you know, social sites and oh, have you checked this website out? Check that out and stuff like that. So I think we'll see more of that. Will probably see the equivalent of you know, a global library, Google esque type thing where you can teleport into and say I want to go learn about medieval art or I want to go learn about Cajun cooking or whatever. We'll go down that aisle, there's over there and you know, maybe a little bit more things like that than sitting. Just like chopping, chopping, chopping. Now the consistency part of your avatar or your environment or things like that, you know, those types of things to your example could easily happen today in the context of a web where you, you go from one website to another website. Or maybe you, you know, we've seen people show up on video conferencing that oh, turn your camera off. You don't realize, oh gosh, or that other one, that guy, that was a, some court case and he came in as a cat avatar. Like I can assure you I'm not a cat. Those things will happen, right? Some by chance, maybe some by design, but that kind of stuff just is the natural evolution. People will know what they should and shouldn't be bringing to a virtual environment. If I'm going to go to a virtual meeting, you know, with my colleagues and stuff like that, I'm going to be darn sure that my virtual presence is going to be business savvy and focused on that. Yeah, but the underlying current around all of those things is consistency. And that means that however you draw the sunlight in the sky, however you experience a physical property in a virtual environment, however you interact with something in that space has to be consistent across the different virtual worlds. Otherwise the connectivity breaks down. And we're back to the early days of the Internet which. Have you loaded this? Have you loaded that? Everything's different. The disconnect will kill the opportunity for Metaverse connective tissue and underlying plumbing, just like we have with the web. That's what's going to make it work.

Yeah, and Omniverse is focused on making that plumbing, making it that underlying layer.

We are based on USD universal scene description which is a format supported. It's an environment supported by many, many companies, from us to Apple to all sorts of companies. And most of the software products today are either have the ability to support it today or have announced publicly that they will support it soon. That's our HTML, that's our common leg. Now, what we do with Omniverse is we're designing a platform that's open and extensible. We don't replace existing workflows, we don't tell you that there's a certain way to do things. We extend your existing workflow and we enhance that existing workflow by allowing your existing tools to connect into Omniverse and to excel with the capabilities that the platform gives it and be able to do more things with it that you wouldn't have been able to do before without telling you. You got to stop what you know and learn something new more. It's like, hey, what? You know, I'm going to give you more to the party of what you can do with it.

Yeah, I think that that's really amazing what you're building with Omniverse. And related to the example earlier of the various use cases that you have for the Metaverse, where do you see the debate going for the open, decentralized metaverse versus the closed walled gardens? Do you think that the enterprise version of Metaverse will be more closed and the consumer will be more open, or what role does Facebook play in this?

So let me talk more in the generic of an open and closed environment, a walled garden for a Metaverse, unless it's by design for something like an enterprise. Like, I don't think I'll be able to teleport into the virtual world at the Pentagon, but the Pentagon will most likely have a virtual world that they can run analysis and simulations and things like that on. By design. I may not be able to teleport into a design center at a company building a next generation product, but that's by design. That doesn't mean they're a walled garden. That means they're a secured virtual world. Now, a walled garden approach means this is our Metaverse and this is only our Metaverse. And if you want to be in our Metaverse, it's the only way you can do it. Those things usually fail. Walled gardens really do not succeed. And I know lots of people like to point at different companies, oh, they're a walled garden, they're a walled garden. But in reality there's very few that are walled garden. You have the openness of the Internet and you have the ability to go anywhere on the Internet, you have the ability to do things. That analogy for the Metaverse holds true now. There'll be different elements and different keys to entry and things like that that different virtual worlds will have. In much the same way the Internet has, sorry, you need a password to get in here, you need a VPN key to get in here. Or, you know, this is a closed environment that only a certain number of people are in, because we're doing something that's not for the outside world, that is not a walled garden, that's a secure virtual world. And I think much the same way we have secured websites, we'll have that kind of stuff, but just by the nature of how we work.

Yeah, that makes sense. So obviously, so the metaverse connecting the virtual or the physical, bringing the physical virtual, and vice versa. How important do you think augmented reality and virtual reality are for this regard? And how do you think it will change in the years to come?

I think ar, VR and all kinds of scenarios like that are just other elements in which you can experience the virtual world. I can experience the virtual world with this as a window into it. I can experience the virtual world on my laptop, like I'm talking to you now. I could put on a headset and be inside that virtual world in a VR environment, or I could mix those things together in an AR environment. But the common theme there is they're all virtual worlds, and you have many ways with which you can experience it. In much the same way that today I can experience the web in any one of those devices I just mentioned, including my watch and other things like that. So it's really points in and points out and stuff like that. I love what we're seeing happening in AR and VR devices. There's some really great ones that are just getting higher and higher fidelity and stuff like that. But as we all know, we're not all going to run around with headsets, you know, banging into each other. In the virtual world, there will be a time and a place and there will be a reason for it. And if you're an architect and you want to walk around a building, you'll probably do it in a VR environment with a colleague, maybe across the world, and you'll both meet in a virtual environment. You'll walk around, you'll look at things and let's teleport up to the 40th floor. I want to show you what it's like. Oh, wow, this is cool. Things like that.

Yeah, that makes sense. And so Metaverse needs to be created. How important are content Creators and creativity for the Metaverse.

Huge. There will be much more content created in the Metaverse than in the physical world. So, you know, people that own a house in the physical world might own five or six houses in virtual worlds. They might own many wardrobes of clothing, they might have multiple ways of running commerce and cars and devices, et cetera. So we think the opportunity for creators is many, many, many times stronger for them in the Metaverse scenario than in the physical world. That's why there's again, so much excitement. People have to build it and have to build it with tools that they're familiar with. They've got to explore it and they've got to explore it in common ways that they understand and that are connected between all these different things. So that's why there's so much excitement about it. It's like in the same day from the web, you know, the web enabled the corner shop to be a global, you know, have a global economy impact. Just by being on the web. You now removed location as an issue.

Yeah. How important are NFTs also? Blockchain for the Metaverse?

Yeah. So, you know, I think the jury's still out on the NFT stuff. It's certainly not an area that we're focusing our foundation and platform on. It's interesting to watch and certainly, you know, explore it to the commerce site. I think there'll be many types of commerce usability in the Metaverse and just like we see on the web today, I can buy and sell things based on a blockchain type environment through different websites and things like that. Same is going to hold true in the next era of the web.

But do you think that that blockchain will be crucial for the open, decentralized Metaverse or.

I think it's already crucial for how things are done on the web. Right. I mean, it's a level of security that allows you to entrust your financial doings. You know, here you're in Australia, let me just send you some money. And you know, here's my bank account, here's all this other stuff without any protection, you know, so blockchain was derived to help protect and secure very costly financial interactions and other types of things as well in a safe and secure environment. So that the key to that is something that neither side has access to. You're just doing the commerce part of it.

Yeah. So how important and how can brands step into the Metaverse and build loyalty and engagement with the clients, with the customers?

We're seeing lots and lots of brands start to adopt a strategy for the Metaverse from designers and architects and car manufacturers and stuff. They all want a presence in this next generation of the web. So we're already seeing strategies start to evolve from them. It's our belief that every company will have a metaverse based strategy in the near future.

Yeah. Just want to briefly go into the dark side of the Metaverse. What sort of ethical challenges did you foresee and the dangers that could come with the Metaverse?

You know, it's like I say to folks, a paper and pencil can do damage or it can write a novel. It's all on how it gets used. You know, there'll always be things that. That are put to bad taste and things like that. That should never be our focus. It's not our focus. We very much frown upon that kind of stuff. We're focused on the good and what can come from that, as most companies are. So that's the best way we look at that side of things, you know?

Yeah. So, and then how does the future of the metaverse look like, according to you? And what is the role that Nvidia will play in it or you would like to play in it?

The future looks like we're just at the tip of the iceberg of all the amazing things that we're going to see and experience. And our position is to be the connective tissue that brings these worlds together and enables a platform for which they can be built.

Nice. Very important work. All right, thanks so much for your time and input, and it was a pleasure talking to you.

All right, take care.

Bye. Bye.

And that wraps up the conversation with Richard Karas. If there's one thing that stands out for me from the conversation, it is how important this digital twins will be for the Metaverse and what a world of opportunity they will open for organizations. I couldn't agree more, which is why my new book, Step into the Metaverse. I cover digital twins in depth, offering an approach to start incorporating them into your business. If you enjoyed this conversation, don't forget to hit the like and subscribe button on your favorite podcast platform, so you will be notified when any of the other interviews will become available. Thank you and see you in the metaverse.