Technical Requirements to Make the Metaverse Possible with Jerry Heinz - Step into the Metaverse Podcast: EP05

Good day everyone and welcome to the Step into the Metaverse podcast. My name is Mark Van Remmenam and I'm also known as the Digital Speaker. I am a future tech strategist and I think about how emerging technologies change organizations and society. This podcast is part of my new book called Step into the how the Immersive Internet Will Unlock a Trillion Dollar Social Economy. The book is available as an ebook, hard copy and audiobook.

As part of this book, I did.

Nearly 100 in depth interviews with the stakeholders involved in creating the open Metaverse. The investors, the developers, the artists, the collectors, etc. And I've turned a selection of these interviews into this podcast as I think that these conversations are too valuable to leave them on my computer. In this interview I interviewed Jerry Hines. Jerry is a thought leader in cloud gaming, beginning his career in network protocol design for multimedia data transport over low bandwidth Networks. He brought GPUs to the cloud, launched the first cloud GPU based service in AWS, and holds multiple patterns in the space. During the interview we discussed technical requirements to make the Metaverse possible, the companies building the hardware required for it, and how to scale the hardware to make immersive events with hundreds of thousands of.

People in the room possible.

Enjoy the conversation.

In one sentence, what is the Metaverse to you?

The Metaverse to me is the next generation experience for communication and collaboration and frankly, human interaction beyond what we've classically termed as the Internet. It's the obvious next evolution to all of these technology pathways.

Yeah. So, and if we have achieved this next stage of the Internet, there's some call it also the immersive Internet. And you know, the Metaverse can be as pervasive as, you know, energy or the air we breathe. How are we going to make this possible from a technical or hardware perspective? That's your background obviously. And so how are we going to do that?

Sure, yeah. So from a hardware perspective. Right. My background is very much in the distributed systems and bringing these all together, we can see the glimmers. Today you look at compute and graphics resources and in network transmission, but really the thing that's going to pull this all together is advances in real time communications to be able to get the things that are generated on those computer resources and transmit them and get them in the hands of actual users. That'll take reinvention of what we've classically done in packet switch networking. The way that all of our communication systems, big or small. Right. If you look at how digital communication is even inside of the same machine there's all these messages passed back and forth. We need that on a scale that we've never seen before. And the distances only get longer. If you want to link people around the world together in a real time simulation such that it's you and me sitting together in a room.

Right.

And really feel that then we need to be doing something different because what we've got today is just too slow.

Yeah. And then do you see the first steps into that direction already happening to developments? Or is it still like really like five, ten years away before we reach that stage?

We're getting there. So, you know, I mean, thanks to Covid, you know, we started to see, I mean, it took video conferencing, technology was around for how many years and now.

Right.

And it took, it took a global pandemic to actually make the darn thing work fairly reliably. And even as we're talking. Right. It's a little skipping on my end. And so the interest is there now where I don't know that it necessarily was before, you know, remote management of applications was something that was something the systems administrators did when they were trying to, you know, work on their cloud server or their data center that's thousands of miles away. Playing a cloud streaming game was something that you did as a really kind of a cool parlor trick. Right. And you know, something fun. It was something that I didn't need to do. Right. Because frankly I had a, you know, a console that could play the game significantly better. Right. But now there's interest in it because I see, okay, wow. Real time collaboration and real time interaction is fun. Right. And it's fun doing it on a massive level. Right. So we're starting to see some of it and you can see it, you know, at the hardware level, Nvidia and how they brought the network closer to and started network the GPUs together. So that way they could operate much, you know, the smaller. Could operate together much longer, larger. The new radio technology, 5G rolling out. It does help to reduce latency quite a bit and keeps that in mind. So we're again starting to see the glimmers of it. But there's massive shifts that will eventually be needed and the big changes will come when the backbone providers and the really large network infrastructure providers start taking notice and start applying all of the research that we've done in the quality of service and routing for video and real communications and start embedding that into the hardware. That's when we'll start to see the biggest shifts.

Yeah. And so which are some of the companies that are actually building this hardware that's required for the Metaverse?

You know, Nvidia, amd, the chip companies are kind of starting to lead the charge. You know, obviously Nvidia is most well known there. And you know, if you look on the network, on the network side, Mellanox, before the acquisition of Nvidia, was moving in that direction, but we haven't seen, and frankly, I mean, hardware is in a lot of ways reactive. Right. It's not an area necessarily of significant innovation other than, hey, we want to make this faster, smaller, you know, try to keep pace with Moore's Law and et cetera. Right. It really. It's the use cases that drive it. Right. You have a really new experience that customers really love. Well, and that can be brought down into the hardware then. Sure. If you look at the evolution of GPUs in the cloud and data center grade GPUs they came from, people were starting to play with it because they wanted to do graphics or they wanted to do AI or machine learning, and these processors worked really well. So therefore evolve the chip set in order to do the, in order to run the workload as an efficiency and optimization. Right. And I suspect that's going to continue to be the way we'll look at the roadblocks of the world and then building the experiences. Right. And then the chip manufacturer is saying, okay, if I want people to continue to use my hardware in the future, then I'm going to adapt to those use cases and help them out because it's a nice virtuous cycle there.

And I can imagine having these massive interactive live events where you have millions of people, but subdivided in like small servers of hundred. Scaling it up to having 10,000 people on the server, 100,000 people in the room, that will drive this as well, I guess, right?

Yeah, exactly. I mean, look at what Epic's done with. Fortnite is also the canonical example here, right? Having concerts in the game, all of those classic experiences. And game developers usually get around that by sharding the environment. So, hey, I've got this manageable world, but you and I are on the server and we're approximately located with one another. And so therefore we can give the illusion of that, but really like pulling in a million versus a million on a scale that's. That's orders of magnitude more difficult, right? You can't do that in classic sharding technology. It just doesn't. The model just breaks down.

Right.

And so the more experiences that are out there, these massive crowds playing in the game experience, you know, frustration. But game designers pushing the experience forward will increase the demand for the improvements. And it'll happen again, happen at the application layer. First we'll drop down into the infrastructure interoperability layers and then it'll drop down into the hardware, as you know, to really accelerate that.

Yeah. So. And in terms of the open source community, the open source community has always been important, I think, for the Internet. And how important is the open source for, for the Metaverse?

Very important. I think, you know, maybe that's never been at a, you know, hardware layer. Right. I mean, hardware. It's interesting. Right. So at the hardware layer, typically the proprietary protocols built close to the hardware are going to get you the maximal performance and so they tend to excel. Right. And so, you know, when you look at a hardware vendor, like you look at the software and the firmware that they provide and as almost a platform that you build upon. Right. And Compare, you know, OpenCL to CUDA, for example, and you just, you see the adoption, it's all Cuda. Right. But in terms of interoperability, right. That next layer, the ability to, hey, I can take these things off of the shelf, wire them together, so that way I don't have to worry about all of the basic interoperability. I can get the compatibility, get the APIs and everything, and I can just focus on delivering migrate experience or, you know, migrate UGC or anything else right on top of it. It's going to be really important. It depends too philosophically where you land as far as who will win the Metaverse. Right. I'd like to think that it's a democratized experience, that there will be a high degree of interoperability between different properties and maybe it's multiple worlds all kind of networked together and I can move between them. As a user, that sounds like the most fun to me and the most engaging. But there are others that believe that no, these are going to be siloed and then everything will happen within that silo. In that case, that interoperability is less. You don't necessarily need it. You're going to have one or two entities that are going to dictate this is the rules of engagement and how you're going to go inside. And so there will still be innovation and open standards within that. Right. But not to the same level as a truly democratized metaverse.

Yeah. So you almost get sort of the Oasis and I guess that's not what we want. We want to have this open, decentralized space, I assume. Um, so. But obviously, you know, in the current website incident, we see a lot of cyber crime taking place that definitely won't stop or get any less in the metaverse. Having, you know, people copying your avatar and using that, or stealing your NFTs or hacking the metaverse, which is for sure going to happen. How to go about that? How can we limit the impact? How can we reduce cybercrime in the metaverse?

Oh, my gosh, I've been pondering this a lot, right? So if we look back in 1998 and thought about, could we predict the Internet security problems of today, right. That ransomware would be as rampant as a thing as it is today and as easy to do. I mean, frankly, ransomware is a service, right? And who might have thought about that? We do have a unique opportunity at this phase to look and say, you know, what we've learned a lot over the last 30 years and how security evolves and how threats and attacks evolve. And so we've got a distinct opportunity to imagine that for the future. And I think it falls down into two, you know, two main areas. The first is supply chain, right? If you think about supply chain, take the natural analog of today, right? So, you know, factories and others that are producing materials that go down the line, right, that are the, you know, the very prominent attacks of today. Well, the supply chain of the future, the supply chain in the metaverse, are all of those interoperability layers, right? They're all the companies that are producing the components that you need in order to build up and create that massive engaging experience. And so you've got to protect those companies. So cyber insurance and cyber security as it exists today will need to continue to evolve to protect those entities because you can't have supply chain attacks. And so I'll have virtual consequences, the magnitude of real world consequences. And then you're going to have the individual security, right? You know, today I go to the bank and the reason why I put my money in the bank is because I can trust the bank is insured. And if somebody, if a bank robber breaks into the bank, then, you know, I will not be harmed, I'll get my money back. And I need. Well, crypto doesn't work that way. NFTs don't work that way. That's fine. We need to invent ways to handle that, right? Because at the same, you know, at the same time it's. While the payment structures have changed and evolved, the problems can almost thrust us back into the Middle Ages. It's Like I'm walking around, I've got this big purse and I'm trying to get from town A to town B, right in, you know, in middle aged England and I get. And the brigands attack me and steal my purse. Right. Of course I would much rather have the guy who's got the giant safe that I could put the thing in and just hold it. And I'm not saying it evolves that way, but this is a real world problem that we'll need to front because you and I will be transacting in this. We'll have real virtual property and that's going to need the same level of protection as our actual world counterparts.

Well yes, because especially I think you can't rely on average John Doe to know how to deal with private wallets and cold wallets. And people lose their passport or their phone all the time. Imagine you're losing yourself from an identity type to your avatar and your whole life is gone.

Right, Exactly. I look back and I remember we were trying to push email encryption and it's like try to explain to your average email user, okay, now you're going to go generate a certificate, you're going to attach it, you have to pass a piece of that to your friend and you can only send to them encrypted. And then it broke people's mind, the average user. Like it boggled security in engineers minds, let alone your actual, you know, the people who would truly benefit from having it. And so all of that is to say like those are great viable solutions today but they need to be simplified. It needs to be common sense. Hey, here's how to do it, here's how to secure. Here's a secure. So that way, frankly, you know, the person who owns the flower store that's selling the NFT of their, you know, like hey, I bought a bouquet and I can bring that into the experience, right. And give it to, you know, my mother who is also in the virtual space and we hand it off. Right. Like we need to be able to transact as seamlessly as we do today and not be worried too much about oh my gosh, is that going to get hijacked? Am I going to, you know, I'm going to have problems there, right. Is the value going to be maintained?

So in terms of, you know, the email example, that's quite interesting because with email I think we've done something really wonderful when building the Internet is that I can use Gmail and you can have ProtonMail and we can connect with each other the same Thing we have not been able to do for social messaging, for. Sorry, for mobile messaging. Because if I have WhatsApp, I can't send a WhatsApp message to your signal. How are we going to fix that in the metaverse for both conceptual and the technical. And is it fixable at all?

Sure. Well, it depends on your view of what the metaverse evolves to. Right. If it's the OASIS example, it's really easy because they could just mandate this is the protocol, right? So let's say it's, you know, it's Facebook meta, right? They can just mandate this will be messenger, right? In that world, you're using messenger, and that's it. That's a communication pattern. It's got a protocol I can plug into it, I can use it and go, that's fine. You know, if it's a democratized metaverse where many different virtual worlds all network together, then we're going to need the email solution, right? It makes the communication more complicated. But look at the. But, you know, I mean, we've done it. We've proven that we can provide even a pretty elegant solution. And also, you know, messaging is evolving that way into more open standards. Right. On my Apple Phone, I can have, you know, I can communicate in many different protocols, right? Through the same singular messaging app, and it's transparent to me.

Right.

You know, it doesn't matter what, you know, what somebody has on the other end. I can just communicate. Right. And I think we'll continue to see that evolution if we're going down the democratized path, which I hope we are.

Yeah, I hope so, too. That would be, I think, the best way. So what's the role of blockchain and distributed ledger technology in the metaverse? Is it predominantly when we have an open metaverse, or is it also the case when we have a closed metaverse? And what's the best direction there?

I mean, it's interesting. Open metaverse, right? This is one of the ways that we communicate across one another, right. When we make sure that it's contract enforcement and value protection and it can fuel commerce at a digital level. Right? Now, inside of a closed metaverse, again, you don't have to. I think it was Second Life had a full economy. They didn't do blockchain. Right. It was database technology that they had, and they did a really amazing job. And it was seamless. And so, again, closed makes it a significantly easier problem in some ways, but then it can block adoption. Right. The democratized version requires additional innovation and advancement to make it happen. But I mean, get into cryptocurrency and, and blockchain. These are quickly emerging as the standards to underpin all these transactions. And they're easy to implement and they've been proven to have durable value. So it's a good stepping stone.

Yeah, that makes sense. And so what role do you think cryptocurrencies will have in the metaverse? And having tens of thousands of cryptocurrencies, do you see a role for all 10,000 of them or what's your idea around that?

I mean, probably not. We're having this explosion right now because each crypto, it's like their apps and other software. Right. So there's each solving a specific problem. And we'll likely see some sort of consolidation down the line. Right. We have to have the innovation explosion and then we'll consolidate down to a few. But if we use the real world as an example, right. I mean, look at fiat currency throughout the world. Every country's got their own, and so it's not hard to imagine a world where different virtual worlds and environments all have each have their own. But the difference is that there will be some sort of currency changer today. If I go from one game to the next, if I buy things in that one game, I can't cash out and then take that into another game. And that's a key difference for when we envision the metaverse or what that experience looks like, is that I can take it, it's property that has value and I can transfer that between worlds. And so therefore, in that case, yeah, the classic money changing. I need to change my USD to British pound and vice versa. We'll likely have that sort of thing, but I do anticipate there'll be some level of consolidation around as we figure out what the best solution is.

Yeah. All right, well, thank you so much for your time.

Wonderful.

All right, thank you so much.

And that wraps up the conversation with Jerry Hines. If there's one thing that stands out for me from the conversation, it is how important the open source community will be for the open metaverse. I couldn't agree more, which is why in my new book, Step into the Metaverse, I discuss how we can create an open metaverse and what the role of the open Cisco community is in achieving that. If you enjoyed this conversation, don't forget to hit the like and subscribe button on your favorite podcast platform, so you will be notified when any of the other interviews will become available. Thank you and see you in the metaverse.