The Augmented Flashlight

All right, so who are you guys?

Hi, my name is Arvind Sanjeev. I'm the co founder at Lumen World.

And my name is Matt Visco. I'm another co founder at Lumen World.

And thanks for coming to my house and showing me some new toys. Augmented reality. This goes beyond augmented reality. It goes to ubiquitous computing.

Certainly. Certainly. Our main thesis is that we believe that the future of computing should be shared and social. And so we wanted to create something that could deliver group experiences. And so that's where we landed on something like this, which is built off of a flashlight, a tool we all know and love. And in some ways it's kind of the first form of augmented reality. We've all created shadow puppets on the walls using flashlights. And so that was kind of the seed that got us to this.

Yeah, so this is Lumen. Yeah, Lumen is the first augmented reality flashlight that brings people outside headsets and lets you focus on the real world.

So.

So this is a demo app that we built where it lets you draw in space. So for example, I can draw a cute heart over this really cute family photo here. And it remains locked in space as you can see. And you can use this for any kind of like digital gravity. Imagine walking in the street in the evening and just like painting the facades of building using the stool. And you and your friends can then later come back and see it through either using the Lumen device or just using the app. And it's definitely turning the spray can into almost kind of like this AR device that you can paint the world with light.

Can you leave videos on there? Because that would be useful in a factory.

I can show you a second demo that exactly does that. So this is our own version of a storytelling app. It's no code based storytelling, so you don't have to open your laptop or do any coding at all to do this.

It's a very literal translation of the flashlight in this one. Exactly. Without adding any content, it's more or less acting as a flashlight.

So in this app, what you need to do is you just need to click on the add asset button and it immediately opens the phone's photo library. And I'm just going to pick a normal video that I can put on the wall here. So here there's like a cute butterfly that just popped up on the wall and you can flying around the family photo so that each time you see this cute animated butterfly, like going around this picture and it stays locked in space like you can see. And in the Same way you can add any other picture or video, any asset that basically lives in your phone's library and just like lock it in the physical space, add and lock digital content into a physical space around you.

What's really powerful about this tool is that it's just very easy to get content in suit like in suto authoring. So you can, you know, a teacher or a museum curator or someone at a factory could very quickly pull content into their space and show that to anyone around them. And you can also use it as more of a prototyping tool. So if you have a bigger experience that you want to create, you can very quickly place content on walls, on floors and get a sense of what it might feel like even for people who are developing things for more. Screen based devices can use lumen to explore what an experience could look like together in a headset, but in a shared way.

So this is an example.

How much is this device, by the way?

So right now the developer unit is going for $1,500, but we are only targeting B2B. We are offering free developer units to creatives who can come and submit a cool use case that you want to work. So you'll be giving out free dev units to them. But then once we hit scale, this will be in the market for less than $250. That's what we are aiming at. And right now we're trying to raise funding to help us reach production scale.

Yep.

This is another example of a small asset that I just placed. It's a small key, just like adding to what Matt was saying. Imagine you're an escape room creator. You can easily add like a digital key that the players can try to find in the space and that unlocks one particular part of the puzzle and then it takes you to another part. So this is something that you can quickly prototype in real time in the space that no other traditional XR device can do this for you.

I think it's also show maybe the wayfinding moments too because just as we can add content on the walls, the ceilings, we can also add content onto floors. And that acts as a really nice moment for you to imagine following some line or footsteps and discovering where content could live in a more spatial manner.

So for example, have this trail that you can have the people follow. If you are in a museum, you can have people like follow this magic trail or follow footsteps that take you to a particular painting and have you reveal another part of the story. So it also helps you in these kind of way, finding Experiences as well. It basically turns the whole world into a canvas for adding digital stories.

And one of the features we're really excited to interact with is, I think maybe this will be a good one to talk about it with, is to make it so it's not just the holder of Lumen who is interacting, but actually both the holder and the group around him. So, for instance, with this one, if this is a game based on Studio Ghibli, the Japanese anime director and studio, where you're basically, you have all these suit sprites that now live on your wall, and with Lumen, you're supposed to capture them. And if, you know, one of the things in our roadmap is for us to be able to use gestures. So I, for instance, with Arvind, could collect these and then help him capture all of those sprites, as well as having it be more environmentally aware. So this sprite, for instance, could bounce off of this light or get trapped inside of this. This painting or this photo, rather, and be aware of maybe where they would like to hide, for instance. These are very skittish creatures, so maybe they want to live kind of hidden away or underneath beds.

So you can see a count of all the sprites that I'm capturing as well on the screen. That's how we also utilize the phone screen. At the same time, you see a counter of all the creatures that you caught. And also in terms of the user interface, there's a small wireless button that I'm using as the interface on the handle here. And that's what I'm using to click and kind of like capture all these creatures into the library. One of the other cool things about especially this app was it was made by a creator called Ollie Taylor. So he was able to build this whole experience in like less than three to four hours. We gave him a developer unit, a free developer unit, and he was able to build this whole experience, our device, in such a short amount of time. That is a proof of how fast the content can be generated compared to traditional XR platforms.

A lot of that is because we've built it off of phones, which is a model that people already know how to develop for. So it's essentially creating a secondary display for a phone. And so anything. Any experiences that live on mobile AR can easily be ported over to our.

Yeah, you put your iPhone in here and you have the 3D sensors exposed.

We're not building any of that hardware. We're letting phone manufacturers, Apple and Googles of the world handle that.

Exactly. We started off with really crazy prototypes before this that used Intel's RealSense tracking and depth cameras and had like a powerful i7 processor with a 2kg battery, the whole thing powered. But we quickly realized that it's really impractical to bring such an expensive equipment to the market. So that's how we quickly pivoted to using a smartphone, a device that people already have, which helps us bring the BOM cost really low, less than $100, which would help us scale really well, but also put it in the hands of more people.

So that tells me that the projectors are getting pretty affordable.

Oh, yeah, totally.

Because our hardware is so light. The primary cost of this is. Is the projector. You know, the rest of it is just a casing and batteries, et cetera. So we. Our focus for our B2C play is bringing the cost of that laser projector down so that it becomes, you know, a much more affordable device than all of the other headsets that are out there.

Totally.

Yeah. It's pretty interesting. There's a couple here.

The.

The second company to that I've heard of that's using projectors to augment the world or build this ubiquitous computing. Where do you think this field is going to go? Particularly because Apple and others are bringing headsets and glasses. Why do we need a projector in a world where a lot of people are going to have glasses on?

I think for the immediate future, the need is that we're still very far away from glasses being mass adopted. And I think there's a lot of uncertainty about how and when. And in order to really deliver on social experiences, we'll need a much higher saturation. And so I don't necessarily think smart glasses aren't going to come, but for those to deliver really effective group experiences, it's very far away.

Everyone needs to have their own version of a glass in order to even see something like this together.

And not only that, everyone, we need to have a networking platform that allows the Spectacles to integrate with the Oculuses, to integrate with the NRELs. And there's, you know, that networking is also heavily complex and hasn't really been started too much yet.

Adding to that, there's also a very popular realization that's happening in industry where the main problem that's in the way of the metaverse and these kind of spatial headsets is 3D. There's not enough 3D content in the marketplace, or there's not enough 3D creators to create stories to allow these headsets to really shine. But that's what makes us unique. Our whole platform uses right off the shelf 2D videos, 2D assets, pictures. You can download something from the Adobe stock platform and just put it on the wall and lock it in place and reveal it. So we are much ahead in terms of all the XR headsets. Just because we get all these 2D assets out there in the world for free that can be pulled in through Lumen and that helps us one, deliver content experiences that are pretty prolific and we can deliver them today. And two, also deliver group experiences that headsets won't be able to do unless they are adopted at mass scale. And everyone has their own personal headset.

And I think the other thing to add to that too is that devices like smart glasses are still a very one to one experience, as are devices like our mobile phones or tablets. We can create group dynamics with phones when we all gather around a screen, but it's still very much designed for personal use, whereas projection is and always has been designed for group experiences. You have your movie theaters, you have your at home projections. The intention of projection is for everyone to gather around and it has that nature of bringing you together. And then on top of that, it's built onto the physical world and so it isn't. You know, glasses creates this kind of separated layer, whereas our device and projection based devices feels like it's honoring the physical space more than glasses and screens might because there is this window you end up looking through, whereas this is, you know, I'm looking just as much at the beautiful picture of your family as I am looking at the amazing drawing Arvin has written of Lumen. And I think, you know, our perspective is that there's so much magic in the world and we want to honor the magic of the world, not create new layers.

I could see even if two or three of us had glasses that this would be a fun thing to do to reveal what I, what I put in the room.

Right, that's exactly.

Particularly for the person who's gonna resist glasses because there's gonna be a lot of people are like, ah, fuck you, I'm gonna.

Exactly.

That's one of the ways in which using our SDK, we can get ahead with the content game and build an infrastructure of all this content layered over the world. But again, this can easily be viewable on Quest or on any kind of other AR headset, but also on Android, iOS, on your iPad, all of the cross compatibility. So we starting off building a platform that's compatible with all these things from scratch from day one. So it's not going to be. It's going to be device agnostic. No matter what device you have, you'll still be able to have these experiences.

That goes with our intention of using the phone as the core engine. You know, our hardware is not the core of what we're building. It's the social experiences and the software that enables that. And so we're not locked to this device. Over time, as more devices come out, we are excited to expand what we're offering into all of those devices. But this is enabling us to deliver the group experiences today that we don't have.

Could you record a video and leave the video on the wall?

Oh, yeah, yeah, totally.

Because that's an interesting thing. If I was a foreman at a factory and I'm walking around looking for safety violations, I might say, hey, I just found a safety violation here.

I can just show you a demo of that where I just click on the add button and immediately I just pull in a video that I have on my iPhone.

Yeah, I'm thinking of recording a video.

But I talked about this. If you just, you know, open basically a selfie camera, record yourself. The beauty of this is that it's already pointed right at you. So you can leave both video and audio, attach it to whatever you want, and then you have kind of like annotated memories for you to be able to recall what you were thinking, but also to. To pass on to whoever's coming into the workshop next or the office next or wherever it might be.

Yeah, this is gonna be interesting because even in a glasses world, I don't have a camera necessarily that's this focused. Right. If I'm wearing cameras on my glasses, maybe it doesn't really help me focus on the problem. On a machine, for instance, that's going bad. And with this you could really. Yeah, you can go really close and record it and say, hey, this is the part we need to order. Or something like that.

Yeah. And you can even play around with the distance, revealing more information. So maybe when you get closer, there's additional layers that you're diving into. And there's a lot of interactions that we have yet to explore in terms of how you delve through information by that spatial awareness.

Yeah. Well, cool, thanks. Where do we learn more about this? Tell me a little bit about your company. How did you fund this and get this, get it to this place?

So we are pre funding. So far we've been just.

You haven't even gotten friends and family seed money or anything?

Not yet. We've Been bootstrapping and just spending all our time to get here. We also been working part time until the last few months when we actually started working full time on this. So, yeah, a lot of blood, sweat.

And tears put into getting it where it is.

Yeah. But right now we are ready to raise our first round and we've already got a lot of excitement from angels and also a lot of VC firms. So we are trying to close around in the coming next two weeks to take Lumen to the next stage.

And where do we learn more about it?

Lumen World is our website. L U M E N W O R L D well, thanks so much.

For coming to my house and showing me this. It's really interesting to think about ubiquitous computing in a new way outside of the glasses world, because the glasses world is coming. But yeah, you're right, it's two years before we get glasses that are really, really useful for consumers, and then another two to five before it becomes enough people get them and understand them and use them and like them.

And I think your point of, you know, how do you. How do you help those who are still resistant, even when it has gotten mass adoption, be able to be a part of things? And so, you know, we might get to a place where most people do have glasses, but maybe my mom never does, and something like this can still allow her to be a part of, you know, that shared experience.

How light is this, by the way?

How heavy?

Yeah, just fairly light, but still something significant. I think that's heavier than my usual flashlight.

Exactly.

Sure.

It's a problem that we can definitely solve through an equal weight distribution. Right now we've intentionally designed it so to make the 3D printing more effective. But when we actually go into production, the handle might just take in more of that weight. And by evenly distributing the weight across your arm, it'll feel much better.

And this form factor is still very much.

You know, you should put a tripod socket. That way you can put it on a monopod.

That's a perfect inspiration for us. The flare of three legs.

Exactly.

Yeah. I'm using a DJI gimbal, three little motors that are steadying my iPhone.

Got it. Yeah, but totally. We have a family of products that would come out with lumen where you can use lumen for leaning in experience, where you want to draw things. But also if you want to go hands free, you can just dock it on our mount and you can use both your hands to interact with the projection.

And as we scale to other industries as well, the form factor can shift to what that industry needs. For example, in the construction space, they may want a larger projector that's tripod docked, but then have a few of these mobile projectors that are interacting together or in the physical therapy space. One thing that we've explored is, you know, maybe you have these projected exercises, but for them, maybe it would be better to have that mounted to your arm or your leg so you're not really carrying the weight, all in this kind of oddly distributed way. So and, you know, in game environments, the gun setup is something that we've certainly heard a lot about. And various kind of handhold mounts, you have the Ghostbusters idea. So there's lots of forms that this can take, depending on the industry and experience that we're trying to deliver.

Well, thanks so much for giving me a little luck. That's really cool.

Thank you so much, Robert.

Bye.