Spatial Computing Catalyst #12: EchoAR - Content Management for the Augmented Age

Foreign. How's it going, Robert?

I'm doing great. So what's going on?

Good. I'm Alain, CEO of Echo ar. We are a cloud platform for augmented reality apps.

I'm Ben, the CTO of Echo ar, cloud platform for augmented reality.

Recent techstars graduates. And we're here in our lab, New York's Spatial Computing center that New York just invested tons of money to build.

Yeah, it's brand new. There's only a couple of companies here.

It's a diamond in the rough right now. They host a lot of hackathons, meetups. The awe meetup in New York is here and basically they sponsor a lot of like AR VR companies, companies to incubate them here in this space.

Very cool. And it's here in Brooklyn.

Yeah, the Brooklyn Aviard. Another diamond and the rough a little bit.

But that makes it cheaper to start a startup.

Yeah, exactly.

Paying high, man. A half.

Exactly, exactly. And again, there's a lot of space and as we know, a lot of like spatial computing companies need that space. Video capture, 3D videos, all that. They just need a lot of space. And this space has a lot of potential for that.

So. All right, so where do we start with you?

So I'll tell you our story because.

We'Re getting these VR glasses right now, right? Oculus Quest. And that gives you. That gives some people a taste. But most people still have no idea what spatial computing is, what augmented reality is, what VR. They're going to figure out 5,000 insiders, everything. But everybody else is like, why do I need this? And what are the pieces that are going to break down? And I'm writing a book and I'm including cloud computing in that book because I see it as real important, particularly when we get 5G and we get a pair of glasses on our face that has a high resolution monitor and has a 5G radio.

True. I mean, for our perspective, it's really bad. Like our space, like the AR VR space is very front end heavy. Like everyone's talking about, you can't see. But what about all the things that you can see when you're building a website? At the end you need to store like images, videos on the cloud. But what's that for AR, like, what's that for 3D? So we're building that cloud platform for AR and VR. They can like drag and drop a 3D model and stream it to device, to stream it to your glasses. And we're talking about AR being mobile, like Pokemon Go was a prime example. But suddenly we have Things like the Oculus Quest making VR mobile as well. So we have to figure out how to stream those things and how to connect all these devices together. And that's what we're building at echoar.

So when you say cloud, it's also confusing because cloud comes with a lot of baggage. Most people think of AWS, where you open up a server and now you have a server up on the Internet somewhere, and it does various stuff, and they have a whole bunch of services, from databases to other things.

Right.

Why do you. And then I'm hearing about this AR cloud thing, which is like 6D AI is building a 3D map of the world and stitching it together. So definitely it's like, confusing.

Yeah. So let's break it down.

Why I need you.

So let's break it down. So the first part with the AWS part, this is the classic cloud component for, like, web and mobile. But the thing is that we're talking about 3D experiences. Something. The experience is different, Right. You can't move an image or a video on your website, but you. But you can Definitely move a 3D couch across the room. And that needs to be updated on the cloud. So only A cloud for 3D4AR4VR means that there's much more like, communication between client and server. And all the time, so you're moving something that needs to automatically update. And so it's much more like data heavy. And you have all these components like, okay, how do you store a 3D file and how do you make sure that it works on all these different smart cameras? So like, Apple has a file format called usdc. Unity works with obj. So for you as a developer, working with all these file formats can be confusing. So if you store it in our cloud, for example, we also make sure that on the fly, we convert it to all different file formats and then stream it to the device in the right format that it needs. So this is something that you don't have at web and mobile, like the classic AWS model. That's one part. The other part is like companies like 6ai, they're creating like the Ark cloud in a sense. And it's a little confusing term because, like, the cloud part of it is still on the device. Right. We're still using a camera, like a client side hardware to capture everything and then store it. But let's say you have that spatial room captured, whatever, and now you want to put something. And you want to say, put Pikachu in, and now you want to switch Pikachu So what do you do? Like that doesn't cover that action of like switching content, managing content, publishing, publishing content to the space that you've captured. So this is the core cloud that we're building. So after you have something like 6ai has built and you're using their tech now you can use our backend to stream data to it, to put it in the space. So our technology works with like 6ai ARKit, ARCore view for you. Like all these different AR SDKs are almost competing with each other, like finding who is the best client side technology. We're just building the back end to connect them all.

Yeah, and we expect the client side guys to go further and further, deeper and build clouds.

So we're trying to be first to market, obviously with the amazing support that we got from techstars, we just graduated their program and got grants from Y Combinator and NYC Media Lab. I think they're really seeing our value being first market, like building that cloud for AR when everyone's really focusing the front end.

Yeah. And then there's Facebook, which hasn't announced what it's doing. But I'm hearing from inside Facebook and inside Google that they're building metaverses, which are clouds, and they're gonna, they're gonna have their own walled garden basically of games and shopping experiences and stuff. And probably a lot of the converters.

Right.

So you can upload a 3.

So for us this is like super exciting. But like all these different companies, like big tech companies are building all these like either metaverses or building all these like client side technologies that we can hook up to. Because for you as a content creator, at the end you'll probably want to post to all of these together. So using our cloud, which is much more like what's called platform agnostic, you're able to publish once and like upload once and publish everywhere. And that's the main goal.

All right, so let's say I'm an architect or a car designer or something like, you know, a car.

I like the architect mix, trillion polygons.

That I want to put on this table here.

So let's do that. Right, so I'm going to, instead of like showing you a few cool slides.

Is that the kind of person company that's calling you right now? Because, you know, not many people have AR glasses and the people who do usually are enterprise customers. That's why somebody who's imaging an oil refinery or somebody who has a surgery going on, or somebody who's designing cars or buildings. Right.

Yeah, that is so true. I mean one of the things right now that because we're focusing on AR specifically because it's already mobile, like you have an AR platform in your pocket, it's just your phone and people are already really inclined to look at the world through their phone. We have AR filters, we have Snapchat, we have Facebook. So it's not weird to look at the world through your phone or tablet. And companies are really trying to capitalize on that. Like people already have that platform. So with us, and I'll show it to you right now, you can just like drag and drop something. You mentioned architecture, so let's do that. So this is the kind of management system like we've built. You can literally drag and drop an image, video, interactive content. So I'm going to do that. I'm going to upload some fan favorite 3D model of the building. So we're in New York right now, so what's better not to have the Empire stapling? I'm uploading to our cloud. What happens right now is that we take that 3D model, convert it to all different file formats for AR on our backend. Once it's done, I am going to open a sample app here. Again, if you're a developer, you have your own app. This, this is an arcore native mobile example. It streams the data to the device in the right format. You don't need to do anything special. Now let's detect some surface here. So we have this cool four over here and there you go. Empire C building is here with me. Classic example of ar. You can go around it, you can go through it. Super, super simple. But the thing is what happens when you want to change this? You have to have a developer at work, you have to pay someone, you have to recompile, compile the app or build it. So that's the case, but obviously are not anymore. We just go back here, delete the previous entry, upload something else and it automatically updates through our cloud to the device. So let's upload something else. Stain construction. So let's upload a different 3D model. There we go, just uploading to our cloud again. That conversion process, that compression process on our cloud.

So how many polygons are you uploading right now?

So this is pretty small. I took these models from poly.gov.com literally for free and just uploaded them. I think one of the models like less than 1 megabyte. But it doesn't matter. We worked with companies who had.

I'm just trying to get a sense of how fast your system converts all the polygon.

Yeah. So we actually shaped that to our backend. And Ben, our cdo, he did a great job revamping it so it's like super, super fast. I think you can test it with.

Very heavy assets and, and it worked perfectly fine. So I created the Iron Throne from Game of Thrones and it came out perfectly.

So now you just open the exact same app.

I was just trying to ask because I'm sure somebody has a terabyte database of polygons.

I mean right now the usage in ar, because it's not that prevalent. So it's not like insane 3D models. But there are some specific use cases like architecture that have have like really like high poly models. That's our job. Literally using them and making sure they.

Convert probably eggs is the biggest ones right now.

Yeah. And again, just depends on the user. Yeah, that's fine. We're ready for it. Like we're working to make sure our cloud is scalable enough to get the point that when everyone's streaming gigs. And I think 5G is also a component of that like streaming high bandwidth and like the need of like high storage. That's where we're going now.

You don't compress the polygon because if I upload a building with a trillion polygons on it, my little GPU there can't render that many true.

So we are working. Yeah. So we're working on like basically reducing the poly count of 3D models to make them more streamable in a sense that will come out on like later versions. But this is something that we're really excited to be doing.

Like a car. If I upload the entire car, you.

Don'T need to have to. Exactly. And this is again what differentiates our tech from like just a generic AWS or a generic Google Cloud platform. Because something we have to cater those like 3D models, 3D specifications. Because think about it, when you're watching a video and you have low connectivity, what happens? You have a reduced frame rate or something. What is the equivalent for 3D? So we talked about like reducing the polygon, but like maybe there's other things that we need to consider that like other companies should not think about. So that's what we do at EGO ar.

Yeah. Get rid of the polygons in the distances.

Exactly. Real time rendering.

If I'm seeing a car, if I'm only looking at the outside of the car, don't send down all the polygons for the Shock absorbers, Right?

Exactly.

That's a lot of polygons, right?

Yeah.

You can't even see until you want to see them. Right. I want to dive through the car, scan into the shock absorber. Then I want exactly. Your. Your system could be building. It could be downloading those polygons as needed.

Exactly. And also, one of the features that we have is location based upon. So the same way if you want to be able to Linux, Pokemon Go and stream Pikachu in Central park and stream something else in the Brooklyn Bridge, you can do that because you can, like, suddenly geotag your content.

Advertisers a lot.

Yeah. And this is, again, something that is not prevalent at, like, AWS or Google platform. You don't actually have, like, images, don't have, like, an inherent geolocation tagging. But we do because we understand 3D models are out there. Like, they're actually in the real world.

So you can put a geofence on a pico chew.

Exactly.

As long as you're within this fence.

Exactly.

Like, if you're in New York, you get this Pikachu. If you're in San Francisco, you get a different one.

And that's exactly. And that's exactly one of the value props that we give developers, that you build, like, one generic app, but, like, it scales across locations. Like, suddenly your app is different in all these different locations just because you use our backend and you can stream different data based on location.

And I use the term polygon. Developers got that right away. But normal people, like, when I talk about how AR&VR works, they're like polygons. What are you talking about? The system converts everything to little triangles.

Right.

And there's billions of them in this room. Right. That's how you would remake this room.

In the actual world. Yeah, they look actually something like that. Like, you can see that the camera recognizes the floor here, but basically it just, like, segregates into, like, small triangles. Yeah. So this is exactly what you're talking about.

And that's how the GPU works, and that's how things work. And as you get more bandwidth, you can make those triangles smaller.

Smaller and smaller.

Yeah, smaller and smaller.

That means you get, like, more quality and, like, the same way.

Yeah. I was playing a fishing game and my Oculus Rift yesterday, and holding a fish, you know, it's like, all right. And it's fairly low polygon.

Yeah.

Right. But it felt pretty good.

Having a high polygon rate just means it looks less cartoonish and, like, looks really, really real. So I opened the exact same app after switching a few of the content and suddenly it changed. So remember we had the Empire State, but suddenly it's something else. And all we did was press a few buttons. No development, no nothing. And this entire example was all native arcore, for example. But we also mentioned the buzzword across platforms. So if we go to. Let's do a WebXR example. So I'm going to open a browser here, go to our website, webxr. Very similar to, to what we did earlier, but now it's through a browser. Right. You don't have to install anything. That experience also connects to our cloud. So now it changes the data to the browser instead of the phone. Same experience, but like different platform. But for you it doesn't matter. There you go. Sony. You see the same building. You uploaded. Once it worked on mobile ar, now it works on web ar. Now let's take a step forward.

Yeah, that's me. So a friend of mine's building a little trivia app and wants to put all sorts of geocached stuff on the world. Right. And Harry Potter is coming out.

Exactly.

After Earth is another one. The kids are so excited about that.

I'm excited about that.

But you could build a game like that on top of Echo ar, have all the assets in the cloud and then if they need to change you.

You just press conflicts. And think about it. Today, when you're talking web and mobile, the person who's in charge of like your web marketing or social marketing is not a techie. It's someone who updates a blog post. So we want to make that process super easy for AR as well. So that person who's not a techie can just like upload new assets, upload new 3D models and then just scale across all your platforms.

Even can. These are hard coded objects, Right. You're uploading an actual 3D object that you get off a Sketch Fab or something like that. What about if I'm building an AI to generate 3D city, for instance, to build a new.

So as long as you can export it, we can plug it in. If you can export it as a 3D model, we can plug it into our platform and then stream it back.

I'm wondering if, if there's going to be new APIs that your system will call and say, I need a view of a.

Definitely possible.

Or this view of a, you know, an ocean.

Yeah, definitely possible. There's amazing companies who are building other development tools like in our pipeline, if we're tackling like the management and delivery of assets. There are companies who are working on aggregation, there's companies working like generating 3D models. We can definitely hook up like their APIs and integrate it seamlessly to ours. That's again the power of the cloud in a sense, like the real cloud.

This is going to really evolve quickly, isn't it? There's going to be a lot of.

Things that come along again, people, after Pokemon Go, you saw that spike in Google trends. AR is a. People know what that means and people want to see new things in the real world. Like the camera is a new feed. It's true, people want that. I think. What is it? Snapchat has like 400 million sessions a month or something. So that's unbelievable. And it's just growing and growing. And one of the things that we are again trying to push is that real time AR apps are not just prototypes. I'm going to show you a third demo. We used web, we used mobile. Now let's do a Unity based app. So it's a mobile app, but it's based on Unity's game engine that a lot of developers are building apps. So let's stream the same 3D model now, but now to Unity again for you. It's totally seamless. So perfect. The same building is here with us, but now we said we have real time capabilities. You want to change something? So let's go back to the CMS here. Let's do direction equals, right? And now when I click on it, if you look at the screen for a sec here, I'm going to do. There you go. So I just click the button and Sally, it rotates and I can rotate the other side. So this is like a really, really simple example of like real time augmentation off the data.

See, that's what I'm saying. You're going to build an API, I predict soon, so that I can write some code to spin things, do things.

Scale it and, and you can move it around. So you can literally do anything with the API.

Exactly. And the thing is again the power is in your hand and you have the cloud to support. So now when we decided that it turns to the left, it changes on the cloud and it changes to all the devices using your app on all platforms. So it rotates on Unity, but you're using magically perfect, it also rotates there. If you're changing it on the client, also updates on the cloud and that is what the AR community is missing. But actual backend to support all these different platforms out There. And if you come to our talk at AWE later this week.

Well this will be up probably next week so the talk event.

But you'll see like I think they.

Put the talks up eventually on YouTube.

Yeah, so you'll see that there. But like you see like all the other companies there are like either need to build their own cloud or they have just like another like camera. There's another smart cam, they're creating another pair of glasses and we're the only ones who really focus on core, like native backend. And that's what we're like super excited about, building that track. Like everyone's saying like our model.

I haven't seen anybody doing this. I've seen like Amazon do Sumerian, which is a tool for sure.

Just building on their smart camera. So the thing is for us we say like that's our motto in a sense, like don't bend on the horse. Then the track people are building all these different things in ar. It could be magic, it could be be Hollands, whatever, but we don't care. We're very agnostic. Anyone could be the best new thing, the next iPhone if you want. But we're building the track to kind of's coming too. There you go. The more the merrier.

15, 16 months from now, Apple's coming, right? And we can't even buy it for 22 months. But that's why this market is going to change so much, right? We're literally at the beginning of this market.

Sure.

It's like 1970, we just got an Apple II and you're Dan Vicklin, you're thinking about building this machine. There's gonna be a lot of them. How do you price this out?

So we have a very simple subscription model, very similar to what AWS or GCP is doing. You get free credits in the beginning and then the more storage and bandwidth they use, the more you pay. You can sign up for a free API key in the beginning you get a limit like limited AM and storage, but it's enough to get you going and we provide you with the support and the tutorials to kick an app like off the ground if you say. And yeah, it's that easy. I mean for us literally making developers lives easier. That's what we're obsessed with. And that's why we go to like developer conferences and hackathons and all these things. Because like we want to make AR development super, super easy and scalable. Because today you can go to a hackathon, you see people creating a prototype and again but your pharmacy peek at you, that's it. And that's a problem because if you want to see AR everywhere, you have to have apps that are changing, that are changing in real time and that the content is constantly updated.

You don't have any. I might have confused somebody. You don't have an API today, yet we do.

So what you just saw, basically we.

Have an API, things and stuff like that.

So we have an API, then we have different SDKs that basically wrap that API and integrate to all the different platforms. So like Android, iOS magically falling.

I can imagine building a business visualization tool and I want to have your tool actually make things bigger or smaller or spinning.

You can do that today, register for API key. We can set you up in like 15 minutes. We literally host a workshop that Ben just did in Denver. How to build a cloud connected ar app in 15 minutes or less.

Where do I learn more about this?

You can go to our website, www.echoar.xy.

We should cover. How did you get this funded? Your background, where did this come from?

Yeah, let's talk about me for a second. Yeah, definitely. So I did my master's at Columbia at Computer Science specializing in ar, VR and human computer direction with Professor Steven Feiner, seminal researcher in augmented reality. After graduating, opened this startup, then joined as well. We got funded from NYC Media Lab, we got grants from Y Combinator and then we got into techstars which were the first big equity investment in our company. We just graduated. We're going to raise funds around June 1, so be able to look at it. Yeah, we launched a product a few months ago. We already have around 80 developers registered building augmented reality application for construction art, like every use case that you can imagine. And we're again just working to support all of them, just building scalable real time applications.

What, what has been done with your API or your system that you didn't expect when you were starting out maybe?

Oh, that's a good question. So we did an art project that was pretty exciting. So for them, they specifically wanted a web based experience because people walk into an art exhibition and they don't want to download an app. So they open a web browser, you can see AR models flying around them. We got like 300 hits a day for that. And for them it was like so unique that suddenly they have like 3D assets in the real world and like making an art exhibition something much cooler and something much more engaging and like people flock to their exhibition just because they use art tech and didn't have to Be a techie. Like that artist. They used it to strap it up with 3D models. There you go. Suddenly it works.

What other tools are you seeing people use? We're going to Sketchfab next, which is a search engine for 3D art.

Yeah, they're aggregating content.

So I bet a lot of your developers have sketchfab running in one window. You have to do running in a third and your thing running in a fourth.

Exactly. Yeah, that's exactly it. I think the AR funnel says, like, content generation aggregation, which is like sketchfab, but we're tackling the next stage of content management and delivery. So you've decided which model you want to have. Now, how do you get into the device and how do you get into multiple devices? And if tomorrow there's a new device, what you have to, like, make sure that you're compatible with that? No, just use Echo ar. We're getting covered.

Anything else we should pay attention to? Because you're. You're probably seeing it all from underneath all the different things that are happening.

I mean, honestly, we're just like, excited about all these different smart cameras and glasses are coming up, because, again, for us, it's just like more horses on track. We want more and more devices to get people on board this good competition. All these big tech companies are trying to create the best new iPhone, like the best new glasses or whatever. That's perfect. We need that. We need more people using ar. We need more people, people looking at the world through their phone, because this is where the magic happens. Thank you. Thank you so much, Robert.