From MUTEK to Mawari: Building the Decentralized Infrastructure for Spatial Computing & XR

And we are back. Ladies and gentlemen, boys and girls, welcome to the Intercognitive Foundation Podcast. I'm Rich Robinson. I am your eager and humble host, the founding chair of the Intercognitive Foundation, and I am here today with Intercognitive Foundation Founding member, Mowari Network, and the co-founder and CEO, Luis. And Luis has the most awesome— I'm going to say his full name— Luis Oscar Ramirez Solerzano. Wow. It just feels like so badass. I know it might not be used in full form, but I really love it. Founder, CEO, Mowari. And Welcome. Welcome, Luis.

Thank you. Happy to be here. It was long due, but yeah, finally we made it happen.

Indeed. And always a pleasure. And I would love for you to tell your origin story, your path from your home country to your adopted country of Japan. You are a mathematician, a festival founder, an infrastructure builder, a Renaissance man. And I love what you're working on and we'd love to hear your version of your hero's journey, please.

Sure. Well, I'm not sure if I could be called Renaissance man, but—

Yes, I just did. I just called you that.

But cool. Okay. So yes, I'm originally from Mexico. I was born and raised there, but I've been living all over the world. So I've been living in Germany, Canada, the US, Korea, and Japan. But I settled in Japan a while ago, more than 16 years, I think. This is why we started Mawari in Japan. Indeed, I studied mathematics, the kind of math that is not useful for anything, just to make theory. But I guess now in the world of AI, actually, that's the one that matters, right? Mm-hmm.

Who's laughing now? That's actually a beautiful— everything's math. So if you're able to really understand the foundation, it's a great building block for everything, right?

Yeah. Again, my mom was always very upset because in Mexico, you need to earn your keep. And it's like, why didn't you go to full computer science or et cetera, et cetera? You're not going to get a good job. You're just going to be a university professor, which again, in a first world country, university professor is well paid and it's a very respectable position, but in Mexico it's more like just, you're a teacher in, in, in, in the end. Right. And I was like, well, but I love math.

Mm-hmm.

And so I went into an area that, yeah, it's just about creating theory, but this is how now you put together XR, AI, and all these abstract concepts into actually something That is manifesting now into reality.

Beautiful. Beautiful. I bet, I'm sure she's a very proud mom these days now. It worked.

I hope so. I mean, probably she doesn't understand much of what I do.

My mom still thinks I make computers even though he works in computers. I'm not sure, but it's okay, mom. It's all good.

So that's kind of like my formation, but also since then I've been involved in music, in the creative technology industry per se. From one side, yes, I was more on the theory, but on the tech side, it was always creative technology. So yes, in Japan, We co-founded a digital arts festival, it's called Mutek. That's actually where the whole idea of Mawari came into fruition because Mutek, by the way, is originally from Montreal. It's, I think now this year—

Awesome.

This is going to be the 26th year. So I actually got to know Mutek since I was a a kid in Mexico, was participant of Mutek in Mexico. Then in 2015, Takeo and I, our co-founder, were like, something's missing in Japan. And it's like, yeah, probably something like Mutek would be a very good fit, especially with the way Japan digital art is evolving. So the whole point of Mutek is to promote and disseminate underground audiovisual art per se. We could call it music as well, but in the end, it's audiovisual art. And most of the artists that perform in Mutek are very, how can I say, creative in the sense that they even create their own instruments, their own techniques. It's all about experimenting with music and visuals and using cutting-edge technology. So in 2015, you may remember 2014 is when Meta, or back then Facebook, bought Oculus, and there was kind of like this first bigger wave about virtual reality. So all the artists of Mutex were starting to get interested into virtual reality and augmented reality as well. But we started seeing a big gap. Like, it was not as easy as just, you know, pick up an instrument or do something that, that's, as an artist, you don't need to go to university to learn how to do it. And in 2016, 2017, making or building a VR application, it was not easy. It was very complicated. So the barrier of entry was very high. So this is where Takeo and I started with, okay, we need to make a change so that this type of content can be disseminated and distributed faster, a way to accelerate this because these creative people is being right now hindered by just the tools. That was one of the initial motivations to create Mawari. And the second one is that Takeo and I always believed that media will eventually will be all just in 3D. of sci-fi movies in the end and the convergence with AI. So for example, one of the stories that we say actually in our manifesto is that Takeo was like, I want my children to see my mother before she— after she passes out and she get to know her as if she was in real life.

Mm-hmm.

And for that, you need artificial intelligence and you need extended reality. So it has always been that, like about that human connection and getting this technology to be something that is normal for every human being. Because also the other thing that we saw is that a lot of people back in 2017 were horrorized to wear a headset. Oh, I look stupid, or why I'm wearing this, et cetera. So it was a bit cliché.

Beautiful, beautiful origin story. I think MuTech is a perfect fit for Japan. Anybody listening has not been to Japan, I was there last year for a month and it is the best brand of a country in terms of living up to the experience of being in Japan. And I think bringing that festival there and then having that experience inform the direction forward for Mawari. You know, people often say what people do in their weekend in Silicon Valley, people will be doing in 10 years, you know, in their everyday life. And it seems like you really followed something that was your, you know, passion and something that was really important for you. And then That opened up this opportunity. And I know in the beginning, as I recall talking to you, you did something with KDDI around this whole very serendipitous timing of the AR opportunity. And can you talk about that and that sort of pull that thread and bring us to where you guys are today?

Sure. So yes, Japan is a very unique country. It has a rich culture, especially Tokyo is cyberpunk in the end, literally. So, and these influences, it's very interesting that you're mentioning about Silicon Valley. This is something that I was actually discussing at another podcast because Silicon Valley is great in the sense that, you know, you have this culture of fail fast and recover and until you get to the goal. But—

Japan, maybe not so much.

Exactly. Japan, exactly the opposite. In Silicon Valley, that also gets abused, you know, like the lack of accountability. Like, it's okay to raise millions of dollars and never bring a result, just move on, make your next startup. And at some point in time, this has been actually a praise in the end. One of the requirements to get funded was that you actually were a serial entrepreneur, whether you had an exit or not, right? In Japan, just for example, with the driving license, it's not about how many infractions you have. It's a point system that the moment that you have an infraction, you get points deducted and then it's very hard to recover from that. So it's a culture there where failure is not accepted in the end, and which is very problematic for startups as well, because in the end, the VCs in Japan tend to be very strong with the startups and in the end it doesn't work out. But so this, I guess, is the advantage of Mawari being multicultural in the end, that we have taken the resilience of Japan because we started the company officially in 2017. And this, I also applaud Nils and Aki because we're one of the few companies in the AR world that are still out there. And we founded a long, long time ago. Most of the AR companies that were— Long time ago. founded in 2017, 2018 is long dead. So that's why this is something that I was talking to Nils the other day is like, maybe, yeah, the geography has made us very different. And the fact that, yeah, we're not locals, but in the end, we started the companies in a different geography with different cultural values has helped us be resilient and navigate frontier tech because AR in, and physical, and Well, physical AI is really booming right now, but it's one of the toughest verticals per se. It's to survive because it takes a lot of time to get there. The Japan thing gave us that unique identity as a company and unique culture. Now with KDDI, so, okay, so back to 2017, so we started the company. We have always had the same mission, which is distribute XR experiences worldwide. But when it really hit to us is when KDDI actually came to us and to other companies, big companies, startups. They had a very interesting RFP. They were already foreseeing in 2018 what's happening right now, embodied AI. In the end, it was not called embodied AI, but they came up with, okay, so we have a digital human in Unreal Engine 4, is powered by Watson, and is such a powerful application that it cannot run on a smartphone and it cannot run on smart glasses. So we want to stream it. And that was the challenge. But at the time, there was no method to stream 3D content. It's just video, like what we're streaming right now. So there were some frameworks to kind of fake the 3D with 2 video streams, for example, 2 eyes or et cetera, et cetera.

Right.

And that we realized it would never scale. So in the end, we won the RFP, we made our first prototype, and we understood a lot, like also that this type of new content utilize, just like AI utilizes a lot of power, this would utilize a lot of bandwidth. But there's a big bottleneck on the networks. We'd have bottlenecks on GPU consumption, of course, but on the Networks is even worse, is networks are always variable. You never get the same bandwidth. It depends on how many users are in the concentrated area, et cetera, et cetera. So there's never the perfect condition for streaming bit bandwidth. So for example, we can get away with 1, 2, maybe 5 megabits per second, but when Originally, VR streaming was around 100 megabits per second. So you need a very artificial environment to sustain a stream of this kind up and down.

It sounds like a job for a superhero theoretical mathematician who's also an immigrant founder because immigrant founders have incredible sort of resilience. What an excellent foundation for a hero's journey. Of course, it's been like the Odyssey of, which I just saw on 70mm at IMAX on film, and it's, you know, 10-year battle and then 10-year return home. But it's been a 10-year battle, but like, wow, you guys are kind of like an overnight success after 10 years. Like you did it, like you actually did it against, against all odds.

This is our 9th year. So, yeah, so that's the, that's the whole point. It, it did take us probably about 6 years to get the technology to the level that it's working right now. And again, it was, how do we create 3D streaming? I told my— our CTO and co-founder Alex, it's like, we need to find a way to create 3D streaming. There was many times that he told me, you're crazy. Why we should be doing that? That's too deep. And I was like, but I guess we can do it. We have now, I think, 7 or 8 patents already granted on these. And we found a way, like we became these R&D partners of telcos in which their need was, okay, we need to demonstrate that 5G is useful in the end. And we would be licensing our early stage technology to prove that 5G has a place to be. Because again, yes, you can watch YouTube maybe at slightly better resolution with 5G, and now 6G is coming into the picture. But with 6G, the use cases, especially with physical AI, are more evident. But 5G, it was very difficult to— Okay. So What is it going to do to my Instagram? What is it going to do to my YouTube or my Netflix? Is 5G really necessary? So there always needed to be that story that 5G is good for XR streaming, 5G is good for AI processing and all these other narratives. So that helped us because telcos needed to demonstrate this and we needed a sandbox to start testing and building our technology, which allowed us to stay unfunded for a very long period of time until we had the technology to a level to say, okay, now it's time to formalize it, build a platform, build a product and platform on top of it. And that's when we ventured into fundraising. So that's really like the foundation. So we created a 3D streaming technology. And just, and the other point that we understood with this embodied AI is that 3D streaming is real-time rendering in the end. So it's real-time processing. So for example, when you think about the current state of content delivery networks, even this video stream, so the video gets generated either offline. And if it gets generated offline, then the CDNs make a copy of this video in a nearby server to the user, right? So this is how it works, like Akamai, Cloudflare, or Fastly, et cetera, et cetera. Other CDNs cache the content across the globe, but they're just copying a static file.

Mm-hmm.

For live video streaming, it works slightly similar, but in the end it's a printed frame and it's the same frame everywhere, right? But now with 3D, you have 2 things. Okay? So if you have your glasses on, you could be looking at me in a very different angle to what I'm looking at. For example, if we were watching the same thing. You have your own point of view. So a central server that prints the same image, you won't get the— it's not working. So you need real-time rendering that is personalized. Wow. And it makes it much more complicated. Yes. On top of that, also AI. I mean, right now when you talk to your LLMs, it's your own processing unit, right? It's not giving the same answer to you than to the other person. In that case, it was Watson to be connected to the avatar and then talk to you in real time. And to feel that it's in real time, that needed 30 milliseconds of round-trip latency.

Wow.

So that's what we figured out how to make it happen, but that's what we understood. Okay, so content, this new generation of content that is 3D with AI cannot be just distributed as a copy. What we need to distribute is the compute and the application that runs out of that compute. It needs to be lightweight enough because it needs to work at the edge. Some of it needs to be offloaded to, yes, to a bigger server, and then it needs to work on the glasses. So this is a concept also that Qualcomm had that is called split compute or split rendering.

Mm-hmm.

Because depending on the workload and the level of latency that you need, we split the processing of what happens in the experience.

Fascinating. Fascinating. I love it. Wow. And it is so relevant to what is happening right now with AI. It's becoming more and more and more relevant and people are more and more and more hungry for.

Exactly. Because for example, if you ask an avatar or you ask a robot, you expect an answer within less than a second because it's real-time interaction. When you're just managing text and Claude or Gemini tells you, I'm cooking or whatever he's saying, it's okay. You go on your phone into another application or you do other things and you come back for the result. But real-time AI and 3D is very different. It needs to happen within milliseconds. And this is a new challenge and a completely new architecture to distribute this at scale. We understood this and we understood that partnering with a traditional hyperscaler wouldn't get us there.

Right.

We tried, actually, we were one of the first partners of AWS on what they called Wavelength in this time. The Wavelength program was a super low latency edge computing section of AWS. It didn't really scale in the end. And this is, was kind of like the second light bulb for Mawari is we need to build the right architecture and the network on how to make that happen. So that's where the entire community-based part comes in. At the time, that was now 2021, we were looking into what was happening in blockchain and we saw companies like Helium and Render Network to really scale supply Based on crowdsourcing. So I was like, that's the key. Because for example, when we were trying to do our second raise, that was in the traditional Silicon Valley model. They were like, okay, get an LOI with a big player that will need your streaming technology and then invest and put your data centers with GPUs. So I mean, that's a big bet.

Usually, yeah, huge CapEx intensive. You have to get one player to believe in you and that player has to actually serve you and cooperate with you. Now you're dependent upon their responsiveness and them not trying to compete with you or go to somebody else. Yeah, it's huge. But you talked about Helium. I see that you have like Helium co-founder Sean Carey was on your board and you're like, let's be the captain of our own fate and let's decentralize it. Like, that was a bold move.

Yeah. I mean, that's the whole point is actually when I pitched the entire vision to Sean, it was very amazing. Like, we spent like 6 hours, 30 minutes. It was like a 30-minute It was like a 30-minute pitch and we just started like, instead of becoming a pitch, because he, at the time he was at Borderless Capital. So yeah, he was on the busy side, but he's like, no, let's continue talking. And we just started brainstorming. It was like, yeah, sparks, like you say, instant chemistry. It was magical. It was magical from that point of view. And yeah, he has helped us on, yeah, like Helium is not perfect because it was a very early stage. So of course he has given us some guidance on what went well, what went wrong. But the proof is there. If there's enough incentive and people believe in the mission, you can scale so quickly. And that was the key for us because we knew that XR was not there yet to be at the level of mainstream. We're starting to see product market fit now, but in 2021, there was no product market fit for any XR application. So that's why it would be suicide for us to raise a huge amount of money just to buy GPUs that right now, with 5 years down the road, would be obsolete. We couldn't monetize them in this AI era. So we needed to grow organically at our own pace. And this is why also we decided to do it in the decentralized way.

Wow. That's a beautiful hero's journey. I love it. And I've read now that you're able to take, like for instance, Unity or Unreal And then cut that bandwidth by 4/5, like a gigantic— and those guys are already optimizing so much. You're able to really, with the edge nodes, be able to move the needle in a huge way.

Yeah. So, okay. So the whole point of our streaming tech is, okay, so you can have a heavy scene on the cloud per se, on the edge. process it. And then we send that in the way that actually you perceive it real time. And that's also very important because as I say, it's less than 30 milliseconds, but it needs to fit the requirements of existing networks. So in the end, this is why the bandwidth is significantly reduced because we create our own framework to do rendering and streaming in the end, because what existed It was not cut for this because game engines were designed to render on device, not through a network and not through a distributed system. This is the biggest challenge because, for example, let's say mobile gaming or your, I don't know, Nintendo DS.

Yeah.

They work because the processor is wired to the display. So there's ultra-low latency and you have that instant feedback when you're playing.

Sure.

So, but just imagine when you add a cable, you add latency, but then you cut the cable and you add kilometers of distance. So what is being rendered and what the user is experiencing, for it to really feel like it's happening in real time, it needs to be around 30 milliseconds of round trip latency, but that's for the image. But if it's interactive and it's, for example, spatial computing, because this is where we focus, spatial computing as well, and that's part of the split compute.

Yeah.

The spatial computing needs to happen on device because if the scene is not anchored sub-10 milliseconds, your brain will start thinking, okay, this is not really happening here. So there's a lot of technical nuances there because in the end, 3D graphics are an illusion because you're seeing in a 2D screen something that looks like 3D. It messes with your brain per se, let's call it that way. And your brain has a way to process and perceive information. So XR is all about that, is how you process that information so that it feels like presence. And when you add the processing kilometers away, you need to dissect each one of the components that make that sense of presence real and optimize for that. So yeah, this is what we've been doing since 2018, pretty much.

Love it. Amazing. And when you really incorporate the whole XR, I know that things are changing so rapidly. But within that experience with motion sickness between like this presence that you talk about in motion sickness, like how does that all dance together?

Oh, okay. So that's the whole point. More than 30 milliseconds, it gives you motion sickness right away. Ah, with the split compute, if some of the things, for example, it could be as simple as like there's some part of the environment that is rendered locally as sub-10 milliseconds. The entire scene in your brain is locking at 10 milliseconds. So those are kind of like the tricks that you need to mix to avoid the motion sickness.

Got it. Got it. Understood. So something that would be annoying just on a 2D screen, You know, oh, this is kind of laggy and, oh man, I missed, I got killed. And it turns into actual physical sickness with XR.

Exactly. And any delay makes you throw up pretty much. So this is why still all the glasses and all the headsets move from wired to on-site, on-device processing, because it needs to be sub-10 milliseconds latency. Otherwise, yeah, your brain cannot handle it. It's similar to, for example, also in sound, the Doppler effect. The moment that the same sound is split to a certain distance in time, you hear 2, but when it's getting closer, you hear it in unison. So it's exactly the same. It's a very similar principle. The closer you get to zero, you will never get to zero latency. And if we're going to—

Yeah.

to math is just a derivative. You will never get to zero.

Never. It's always asymptotically. There's got to be some sort of lag. But the fact is, as you said, I mean, 30 milliseconds, it's really difficult for me to even think about that because of course 100 milliseconds is 1/10 of a second. So it's so small, but it's fascinating. Like, I was in Paris for this RAISE Summit and I went to the new Notre Dame. And I asked AI about Notre Dame and they said, look at the church. The church is actually completely asymmetrical. The left tower is a different width than the right tower. And these porticos are all different shapes. And they did that deliberately because subconsciously your brain loves asymmetry. And now I can't unsee it, but it's just fascinating about things that really make something very enjoyable and very addictive and very just delightful are things that we don't even necessarily consciously perceive.

Which is, that's one of the points of the digital humans and that's another Big topic, the uncanny valley.

The uncanny valley, yes indeed. For our listeners who aren't familiar, the uncanny valley is probably the robot probably looks like me, but unfortunately for the robot, but it feels too human but not human enough, but it's this weird kind of like oh it feels it creeps me out, right?

And it's also asymptotically because you can never get. to the human. And the whole point is that you can break the uncanny valley, but you will never be human. That's the whole point. But we haven't gotten there to the— maybe AI video is getting very close to lift that threshold. You can still tell when an AI video is an AI video and this is not a human person, although it The person generated by the AI model looks agreeable to you, you can still feel something is off. But when it's further away, it feels weird. It feels very weird.

It's fascinating because somebody gave me a book, hey, I wrote a book, and then I read the first paragraph and I was like, you didn't write a book. AI wrote a book. I can just tell by the way sentences are structured. And then you see the Will Smith with 6 fingers eating spaghetti like a weirdo, but then you see something on Grok Imagine Now and you're like, wow, that's pretty smooth. I guess there is this like sub-30 milliseconds, 10 milliseconds. It gets asymptotically to zero, but it gets enough that maybe we do overcome all of these limitations and things become like Westworld. That it becomes so hyper-realistic that it's indeterminable.

We will get there, but this is one of the reasons also why we work with avatars right now versus hyper-realistic humans because of the uncanny valley.

Yeah. Tell us more about your work with avatars. Man, I'm so fascinated. I love what you guys have built. Wow.

That's a Hollywood trick in the end. Okay. So all the animation movies, None of them are— they can look hyper-realistic. Think about the Avatar movie or any other Pixar movie. All of them are humanoids, but they are not human. That's why your brain accepts them right now. So getting— and this is what we learned with KDDI, that because they were trying to do a hyper-realistic digital human, but we were always hitting the Wow. So that's what we understood. At some point in time, maybe in 5 years, 10 years, we will get to that gap of the uncanny valley that maybe the human will not notice, but we're not there today. And it's many things. It's the facial expressions, the movement of your eyelids, and anything that makes you human. So that AI hasn't figured it out yet. I think it will at some point, but—

Fascinating. And tell us about your future at Mowari and what the roadmap is looking like and what you're excited about.

We have a platform called Arawa, and it's in alpha stage, but it's a Content creators platform. So there's this new trend of becoming a virtual YouTuber, which means instead of just being a normal content creator where you show your face, you do it through an avatar. So you don't show your real face. Of course, there are a lot of AI girlfriends and other use cases, but this one is more like, is you the one puppeting the avatar? Because this is the other thing that we understood when we were doing the Same experience, A/B test, a digital human powered by AI about 2 to 3 years ago versus the same avatar, but being puppeted by a human. Their engagement and the time spent in the experience was night and day. So for example, when you're talking to an LLM, you don't feel that personal connection. But when you're talking, even when you're talking to an avatar, but there's a personal connection, you feel like there's a soul behind that digital presence. Yeah, a soul. You start connecting and you start asking random questions. You start and you forget about the technology. So this is the magic that we have found right now in the product-market fit for XR, which is a digital presence or digital sense of presence. It will get to embodied AI, digital embodied AI at some point in time, but it needs a lot of optimization, just like video models. Right now we're at the stage of the Will Smith with 6 fingers and the spaghetti, like really weird. We're not there yet for these autonomous avatars to be chatting with you like you feel Nothing is off and you just think of them as your friends. We think the right way to go there is not with digital humans, but for example, you can start with the same concept as Tamagotchi. So you can have a digital pet.

Ah.

Fully AI-powered that is more agreeable. And little by little, then yeah, it will evolve into, yeah, your, the Blade Runner fantasy that everybody has, you know, You know which one I'm talking about.

Yes, yes, it is. Of course, you are based in Japan, so of course that has to be considered. That's amazing. You're really— I think about the Intercognitive Foundation. Our mission is to make the physical world accessible to AI, and I feel like there's these 2 underground boring machines that are creating a tunnel And then you're coming from the other side of like, you're really making AI accessible to humans, but you're doing it in such a, like, wow. I mean, Japan really gives you this incredible space and lab because there is so much, you know, it's such a large country and It's so technologically advanced and it's so cultural and dynamic and entertainment and the money is there from the big, the big companies. So you've been able to do these amazing experiments over and over for the last decade almost, and to be able to really uncover things that I don't think there's any other company that's anywhere near close to what you're doing. And it's actually really Really fascinating how you've been able to just solve problem after problem. It's a beautiful journey and it's just, it's like you've been training for 10 years and now you're really, 10-year bootcamp, really a lot of pain and a lot of suffering, but there's all these beautiful things and now you're like ready to go. do this battle in the AI arena. That's so incredible. Tell us and tell our listeners about where you're looking to grow and do new commercial opportunities around the multiverse.

Well, definitely, yes. In terms of roadmap, let's say we have the platform, Arawa. I think by next year we will be ready to launch But we're also very interested in collaborating, working on how to make AI understand the real world, but the other way around, like how to bridge that. Because that's also the gap I see with world models because it's only one-sided and they need the other side. And I think we can fill that gap.

Absolutely. You guys are the one. Wow. Man, I, I really, um, would love to come and visit you and see your crazy secret underground labs, uh, in, in Shibuya and, uh, and, and have me some, uh, Japanese food and soak up some of the atmosphere and, uh, spend some time with some— spend some time with you. Thank you so much for your, for your time. I love hearing about your journey and really looking forward to reading more about you guys in your glorious future.

I appreciate it. And yeah, I mean, we are excited for this new phase in general of Frontier Tech and very keen to start collaborating more closely finally with Intercognitive.

Wonderful, wonderful. Love it. Thanks once again. Put your hands together for Luis, everybody. That was awesome. Thank you. All the best.

Thank you. Cheers.

Adios and sayonara.