David Rose of SuperSight: The Future of AR

Startups get the party. Startups get the party started. Startups get the party. Startups get the party started.

You want tech and tacos? Tacos and tech? You know we got tacos and tech is on smash border green guacamole keeper captains counting beans Contador Quentan frijoles found us trying to raise some queso la vida San Diego school of rappy rapping fire tortillas round jalapenos in these modern times a crappy and coffee my coding yoga posse fold the body origami we be surfing coronado eating tacos El gordo, the taco standing puesto lucha libre and sombrero Got a shout out to cost angels by the ocean watch the sunset la Jolla shores Entrepreneurs trying to get funded When San Diego Startup Week is popping in the summer Oceanside, the Tijuana esti be full of hustlers from couch surfing to penthouses right on the beach Tech leaders believe stars are within reach we interview and founders who be shooting for the moon SD Tecos with your host, Neil Bloom.

Startups get the party. Startups get the party started. Startups get the party. Startups get the party started.

Welcome to the Tacos and Tech Podcast. I'm your host, Neil Bloom, and I'm thrilled to have you here. On the Tacos and Tech podcast. We interview builders of technologies, of companies, of communities, and we do it weekly, so thanks for tuning in. Every Tuesday, a new interview, and on Fridays we talk about what's new in the San Diego tech ecosystem from a news perspective. So make sure you join us there as well too. We also have a tech newsletter that comes out every other week@freshbroodtech.com you can subscribe to listen to it without further ado. Enjoy this episode.

Thanks for inviting me. This is fun.

Yeah, no, I'm really excited to get to know what you're working on. Do you want to give us a quick overview of what supersight is?

Sure. Well, I came from a background in IoT. My last book was called Enchanted Objects. And I've had a number of startups that have been doing work in Internet connected things. So, like Internet connected pill caps to ensure that people take medications more regularly. I had a company called Ambient Devices, which was all about little wireless indicators in your life that would help you kind of know about things that you care about, like whether the surf's up or when the next bus is coming or how many steps have you walked. You know, little kind of things that communicate to you. So I've always been kind of interested in new technology waves. And I was working at Warby Parker in New York and we came out with a, an app that would help you hold up your phone and see new glasses on your face. And so for me, that was like kind of this watershed moment of wow. Like this next tech wave, it's going to mean augmented reality using computer vision and it's going to just unlock a whole new set of services. So that's really what supersight is about, is kind of thinking about how will technology change what we see and how we see.

Oh, I love it. We're going to have a great conversation. I happen to have just used the Warby Parker AR app over Thanksgiving and bought some new glasses that are coming my way.

So what's so crazy? Not to interrupt what's so crazy about the kind of virtual try on for e commerce is you just, you know, we're seeing it in so many categories. Like there are companies that are selling jewelry and helping you like see what it looks like on your neck or watches on your wrist or shoes on your feet or just like so many categories. We thought it was just for, you know, dropping couches in your living room. Thank you. Wayfair and Amazon and ikea. But people really find it. It kind of accelerates how fast you make a decision.

Yeah. And with the metaverse like now, the people will be there kind of in the same way you would have in a store as well. So I think that part it can be interesting to talk through as well of kind of your learnings around that because obviously Warby Parker has mastered the L experience and reimagined it. Take us through a little bit of kind of the history of, I guess, your background, but how IoT is kind of iterated over time towards what you're seeing towards today.

Sure. Well, I graduated with a physics and fine arts degree, like kind of not knowing what did I want to be when I grew up. And my first company made museum exhibit designs for like Boston Museum of Science and Chicago Museum of Science and Industry. You know, really when you could like spec a very expensive, you know, Silicon graphics machine to create an experience that nobody could really have at home, but you could certainly get if you were just doing an installation. And one of the projects that we worked on in my first company, which was called Interactive Factory back in the 90s, was a project for Lego where LEGO was trying to think about kind of how to expand their market. And this was really before we had the words IoT. We embedded compute into LEGO bricks, a big brick that had some batteries in it and so you could hook up sensors that were also embedded in Legos. So you could kind of build little robots that would, like, have a long nose and not fall off the table. So you could build little rules that would say if the nose is not over something, then back up and, you know, go another direction. That was kind of my first IoT experience. I worked on Guitar Hero in those years, worked on a photo sharing startup back before it was a good idea to do digital photo sharing, and ended up starting this company, Ambient Devices, which was really thinking about, how do you take all this data that was converging on the phones at the time and really kind of spread it thinly around your environment, you know, into lighting or into things that kind of looked more like sculpture and less like computing.

Oh, wow. Super interesting. A lot of kind of B2B kind of. It sounded like, you know, four museums, but then eventually to a consumer or something like that. Have you seen that change? I'm guessing, obviously, with Fitbit and others, the direct relevance of consumers wanting to understand their data. Where is that part? Where did that start to take hold? I'm curious, on the consumer side.

Yeah. I think one of the things we realized at Ambient Devices was if you show people data that is collected in a passive way, like you have a sensor on your shoe, but presented in a way that's unavoidable, like, rather than having to go to an app, it's just presented as like a colored light or a gauge that's on your desk. People. It's really a behavior change opportunity because people really respond to, you know, data when it's glanceable. And so I think that's probably still something that companies like Fitbit or others should pay attention to is like, how do you project data kind of passively in people's environments? And that's, like, so much more effective than requiring that somebody goes to an app.

Yeah, that's really, really interesting and astute in that regard because, yeah, our data is being collected in so many ways and not utilized. God, there's got to be so many crazy data warehouses out there about us. So kind of take us through, like, where. What did you learn through that toward To. To direct you towards the next company?

Well, I think I've always been looking for ways to kind of make computing ambient, or what's now called calm computing, kind of ways to embed it seamlessly in our environments in ways that don't feel like tech, but they feel more like fashion. You know, a watch, a shoe. A lamp. I did a series of products when I was teaching at the media lab in a called the Tangible Media Group about kind of pairing furniture with information and services. So we made like Skype cabinetry for your kitchen so you could just open a cabinet and like see a loved one. Facebook coffee tables that would like listen to the conversation you're having and show you images that you've posted online that complement that conversation. You know, doorbells that give you information about who's heading home. So not who's downstairs, but also like who's on their way. Yeah, the New York Times did this nice piece kind of when my, when the first book Enchanted Objects came out. Kind all about these ways of marrying the world of furniture with the world of data services and ways that could be people would find magical. And I'm, you know, I'm still impressed that the companies that do make lamps and furniture haven't done more to kind of embed services in those, you know, to differentiate. Like imagine you had like a Skype recliner and the more you recline that, you know, it picks different tempo of music or different playlists based on your kind of angle of repose or which direction you're facing. Like it would be kind of like a cool, a cool way to mellow out that wouldn't require like immediately getting on an app.

It's true. Why do you think, yeah, like the adoption has taken seemingly slower or just been less creative at the mainstream in that regard?

Yeah, I don't know. I went to the Milan Furniture fair and was just so excited Maybe this like 10 years ago to like let's do a deal with Magis who makes incredible furniture, or Artemity who makes incredible lighting, you know, because like those companies are yearning for ways to differentiate their products, but haven't really entered into embedding tech in them. I think what we do, what we are seeing more of is like the flying experience on airplanes, the driving experience on high end cars. Like those are places that are really being infused with technology services now. But I think it's just a cultural reason that if you're a furniture company like Vitra, you know, you differentiate through materials, not through technology. Embedded services.

Yeah, that is interesting. Especially on the investment side. I think there almost is like a clamor or, you know, a frenzy for what is the next angle? Is it a sustainability thing? Is it an experiential thing? Is it a data thing? And the answer is yes. It's somehow synthesizing everything to figure out what the customer wants, not just what I pay for a luxury item at.

Right, yeah. Well, I think the advent of a lot of the ar, the augmented reality tools that we have that are now built into all of these phones are really providing a lot of opportunity for companies that are making furniture or selling plants or selling boats or whatever you're selling to allow people to configure those things, to drop them into their lives to imagine themselves in those places. And, you know, two of the projects that I'm working on right now, now, are really about using augmented reality for safety and navigation. When it comes to being on the water and in a boat, that one's called Clearwater. And if you think about all the tech that's in a Tesla, there are sensors all over that car, there's a 360 stitch video, there's computer vision that's being applied, you know, that should and actually needs to come to the world of boating faster than cars. Because boating is such a dangerous environment. It's unconstrained. There's fog, there's seas, they're tides, there's wind. You know, it's a lot. There are no roads. Like, it's more like flying a plane than driving a car. Right. There's like a. Except there's no FAA and very little training. So we're trying to bring some differentiation to the experience of boating to get more people out and safer and reduce accidents. And I think AR can really play a role. But for all kinds of navigation situations, you know, think about how many jobs in America involve getting yourself somewhere, you know, or like getting resources, getting to some place at a certain time, you know, navigation, getting people through a hospital system without getting lost and without feeling frazzled. You know, so many companies where navigation is, could use an assist. And I think for. In the book, I talk about kind of the difference, the different use cases for ar, and certainly navigation is one of the. One of the big ones.

Yeah, let's talk about some more of the use case because you've obviously talked built a lot of companies around this and obviously way before it even had a name that it has today. And even today's names, Arkansas and Metaverse, keep people thinking like, oh, some big fundamental change is going to happen where I'm just going to like plug into this thing. But like the boat or the car, it's coming to you, where you are, where are you seeing? Like, I'm just curious, like the nexus there, like, how are people bringing people in to this AR or Metaverse environment kind of on the front lines?

If you want to Sell books or newspapers or magazines, you really want to tell a story which is like it's going to change everything. You know, like self driving cars, they're coming soon, they're changing everything. But if you look at the way these things roll out, it's really about kind of the long nose of innovation where little incremental changes start kind of infusing themselves into cars. Like self driving features are most cars these days, they have responsive cruise control so you don't run into the person in front of you. Like that keeps a safe distance. They have blind spot warnings, those given by like the understanding who's next to you. There's lane keeping which will kind of nudge you over, do a virtual bump, virtual rumble strips, you know, if you depart off the lane. So like cars are getting a lot safer and having a lot of these self driving features, but they're coming incrementally and I feel like that also will be the case with a lot of AR features, you know. Ed Warby Parker There wasn't an AR app for Warby Parker. It was just a feature that's in the normal app. So you're like you're browsing glasses, you swipe up and you get the camera view of yourself in those glasses. So I feel like these things will be very valuable features first rather than wholly dedicated, you know, build from scratch metaverses.

Yeah, it's a great point. I mean I think some of the, you do have, certain ones have to be splashy to say hey, this is going to be a fundamental change. Like Google Glass had to say this is going to be something that's going to be front of your eye. Didn't sell. It wasn't a consumer product in the end, but it at least turned people on and especially the early adopters to be like how could, what are the use cases for this? And now you're seeing like pretty much Google Glass for blank. Right? Or that.

Have you tried Neil, have you tried the Google Navigation AR experience yet?

Oh no, tell me about it.

So it's just part of Google Maps. So you come up out of the subway and you're like which way again? Like what's my orientation? Like you don't have any context. And there's a button now that's just underneath that says, it doesn't say AR View, but it says Vision View or something. Let me actually, let me look. And so you hit that button and then as you hold your phone up, it superimposes the destination, the turn icon on the world above you. So you're looking through the camera of the phone and you're seeing overlays over the camera view of the phone. And it's using vision to kind of of get where you are and what the facade is that's in front of you in order to help guide you. But it's so much more intuitive than to look down and try to figure out where you are on the map and which direction to go.

Finally, about time for sure. Because how much times have we like try to figure out where direction of what corner am I actually supposed to be looking on compared to that really makes sense in that regard. Where I'm curious, someone you've built companies, what are you telling other entrepreneurs and founders who are asking like, where should I be leaning into building? And also timeline wise, like, when should I be expecting to build versus user adoption on some of these AR tools?

Yeah, I just did a workshop in Italy. It was right on Lake Como. It's a beautiful place. Like Clooney's place was like right across the way. And it was companies that were there who were all interested in like, how do we use AR in our businesses today? And so, you know, in addition to navigation, I think remote expertise, bringing people together in a way that also includes context, you know, that also includes being able to see through somebody else's eyes, be able to mark on the world in front of them. That's huge. Like, so if you're a company that has people that have to go, you know, one of the companies there was an energy company and they have to go out and fix issues on the lines and find things underneath the snow in the mountains and, and not all of their, not all of the people have the same set of skills. So if you have somebody in a place like rather than just holding up your phone and doing a FaceTime session, if the other person who's remote can see what you're seeing, if there's a digital twin of the power plant so that they can fly a camera around, if they can mark up things in front of you so they're like, not that thing, this thing, like, here's the sequence of things to do. I think that's a kind of no brainer use case for using AR for anything where there's people that need to connect around information and that information should be spatially anchored in the world. So that's a big, I think that's a big one. And the other, you know, the other one that I just think is, you know, this, the ninth chapter of the book is really about what do we need these new eyes of ours, our evolved eyes to see? And the point that I make there is like there are industries where people are just where people need an imagination engine. Like they need new eyes to see things that are just hard to fathom. Like it's just hard to, it's hard to imagine what your lawn could look like if it were totally sustainable. Right. If you pulled up the grass and put in things that didn't consume as much water. Most people just don't have the imagination to do that. And you wouldn't expect that they would, right, because they're not trained as landscape designers or you walk around a city and you just, you can't imagine like what it would look like to have a more pedestrian friendly intersection here. But with ar you've got these tools that now allow you to kind of see things that are hard to imagine. And I think that's one of the real kind of features of this medium that, that's really exciting.

Let's talk a little about where you see from a technology standpoint, people have to make the investment for their own company to do this, right? And that means maybe a much bigger tech stack, right? Because you're inputting a lot new data, maybe a vision system versus the long term benefit of cost savings. Whether it's hopefully not too much talent loss or new talent training on this. Like, where are you guiding and advising people to kind of focus on around that piece?

Yeah, I do, I do think it's an opportunity for anybody who's a designer or interaction designer or head of product now to kind of level up to understanding a new set of tools. So like when mobile first happened, like we were all kind of clueless about how to do that and how to use all the features that are inside of mobile. You know, how to use GPS and how to use the camera and how to use all kind of social features. With ar, it's like a whole, it's another new set of things to figure out how to deploy and how to design. And I really encourage people, luckily there are a ton of tools I really encourage people to just try Reality Composer, which is Apple's super easy drag and drop kind of ar. We use that for workshops a lot. Adobe has a product called Arrow. So if you already are in the Adobe suite, you can take all of your assets and kind of dump them into Aero and start placing them and planting them in the world. And this time there are 3D asset stores that are in Unity, in Unreal Engine, in, you know, Turbo, Squid, Sketchfab, that allow you to like find other people's 3D assets and drop those in kind of Lego like and build experiences around those. And like that's super fun. It's fast and you can communicate a lot to the rest of your team just in those tools.

That's great. Something you mentioned about you did, an event at Lake Como came to mind that you know, this potentially is going to change travel interactions around a geographic piece. What do you see the future of that? Especially just during COVID we all just got rapidly exposed to the remote only kind of thing where what is air going to do for us in that regard?

Well, I think for physical, I mean it allows you to preview those things. My wife is going to taking a trip to Thailand and is able to like put on the Meta Quest Pro and like kind of drop into those places where she's thinking about going. And there's an app called Gather which allows you to kind of pick places. It's kind of a little Google Earthy and so you can kind of get a preview of those things in. In kind of a spatial way. I think for the venues themselves. A lot of towns and national parks already have a lot of kind of signage around to interpret the world. I think that opportunity is huge. The scrub through time, you know, see the ruin, ruins rebuilt, see what the, what the parade felt like when you for Mardi Gras when you weren't. Even though it's not Mardi Gras today, you know, just make places more. More active and more vivid and kind of explain the history and explain the personality of the place. I think there's a huge awesome opportunity around for that. But I also think just in terms of, you know, us all working remotely more, I'm super excited about these companies that are really focusing on how to spatialize critiquing a product or certainly if you're like in architecture or interior design, getting everybody to not only fly through it on their own, what something could look like, but kind of gather in places because there's so much, there's so much body language or what I call in the book kind of the gaze vector. Like if you can see what other people are interested in and attending to or frowning at or whatever. Like you just learn so much more than having a critique that is, you know, that isn't in the place. So I'm like, I'm excited about importing these. Like one of my students from the Media Lab has a company called Spatial. It's Spatial IO and you can kind of import any FBX file if you're working on a product or, or an environment. And then you could, and then everyone can jump into that place and you know, have a conversation about the design of it, which I think is so much more powerful than just everybody looking at computers.

Yeah, really interesting. You mentioned something earlier about, you mentioned Digital Twin and I thought maybe we should unpack that a little bit just to make sure the audience knows what that is, because it's going to become pretty standard piece in this AR world. Like where you see what are digital twins and where it's their special feature that are going to enable us in the future here.

Right. So the history of the term came from when NASA would send up something into space. They would have a second, a digital second version of the capsule or whatever it was that was sent up into space so that they could explain that to people who are off site or they could think about how to reuse pieces of it. Today digital twin just means a digital copy of the model of the, whatever it is, the chair, the tree, the power plants. But it also means like a lot more metadata about that thing, which is when it was serviced, how to service, what's likely to fail next, who has which skills in order to be able to manipulate the thing in different ways. So it's like it's now kind of taken on a lot more layers of meaning than just a 3D model of the thing. But what's exciting about digital twins, I think is the, the new tools that like my favorite, one of my favorite apps is called Polycam that allow you to scan and quickly create a digital twin of a place. Have you done this yet?

No.

Oh, you should download Polycam and just like walk around your house if you have, especially if you have a lidar phone. There's something called Room View and it show like, so you look through the phone, you see the camera view and then all of a sudden it starts to draw these like bright lines on the boundaries of your windows and desks and the corners of your room. And then right in front of you, like build a little 3D model of the room as you're walking around the room. So rather than having that be like a two step process like it is, there's another app called Canvas which does this. With Polycam you can like really see it being built right before your eyes. That's a beautiful way of creating a digital twin of a space. And then you can drop it in at like, like breadbox size as like a dollhouse thing or as a one to one view. And it starts to get pretty freaky. Freaky. If you're like in Lake Como, we had. It was in this beautiful villa. So I did like a polycam of the downstairs and then brought it upstairs and put the upstairs downstairs and. And then showed that to people. And it's kind of uncanny if it's like very close to what you're currently looking at.

Super interesting. Something about, you mentioned about the data aspect around capturing this. Right. The meta, at some point it interacts with human data, our own data. Right. You mentioned service records and other things like that. Like, where's the boundary of where we will. Maybe this is too much of an ethics question. But like, when does the human aspect of the AR meta piece, like, if someone's capturing us inside a model without permission, where does that. Where do you think that's starting? The conversation is right now.

I just had a, I had a coffee yesterday with a friend who's doing face reco for ticketing at sports venues. So maybe this will get at your question a little bit. So he was saying, look, even though it feels a little bit creepy to have the face be your pass, to be your ticket or your key into a venue, it's much safer. It's much more. It's much faster. Like, there's a lot less friction. One of the reasons it's safer is because you don't want crowds to hang out outside of stadiums. You want them to actually come in because at least there's a, a check or a filter for who's around. So it's safer to be like kind of inside the stadium rather than hanging out outside of whatever the venue is. Also, the venues want, like, people inside, eating and drinking as soon as possible. So there's a business model there. And you can't spoof a face, like with his system in the same way that you can spoof, you know, you can like hand somebody a key card or hand somebody a ticket. So it's a lot safer. So it's faster safer. And then there are other reasons it's better. But it does bring up this question about like, well, still though, like, who's got the data about all these faces? And like, should they. Should we do that on college campuses? What about, you know, he. And he tried to say, look, there's a bright line between surveillance and looking at faces from afar and faces as keys.

True. And obviously we have. There's databases that have a very personal information now, 25, 30 years that have not been shared. Right. Passwords and many other things.

So.

So theoretically the system has upheld, you know, personal information. Yes, there's been data breaches, but it's, yeah, it's this next layer of, you know, my face, my person. Maybe next is aspects of my personality, my likeness that are being used in ways that you didn't expect. So that is really interesting. I think you, the cost benefit analysis that you make there is kind of interesting point though.

One of the signals that I'm really interested in when you talk about kind of the digital twin of people is engagement. And like when you're like, I do a lot of teaching and I'm always, I always love being in a room with people because I just get so much kind of subtle signal on. Like are people interested in what I'm saying? Am I talking too fast? Am I boring anyone? I can adjust a lot about how I present where I stand. You know, if there's somebody who's kind of not engaged in the back of the room, like I'll go and stand behind them or ask them a question, you know. So all of that signal of engagement I think is really something that could be powerful to mine for teaching and training situations. Certainly entertainment, right. Like you go to a concert, like you want to see the people are into it and you can probably do a lot of. A performer can do a lot in terms of engaging their audience if they just knew what their engagement level was. So to me that's like one of the more positive and exciting signals to think about.

Yeah. Like you mentioned from a technology company perspective, like this is really is incremental. Right. So like the fears and things that we envision are just like the same, like flip a switch and boom, everyone's on some brand new world. But this is incremental and I think that's good that technology is incremental.

I think it's also going to be episodic. Sorry to interrupt you, but you know, it's like everyone who's, who's judging the hardware for AR glasses and like, like if I have AR glasses that do translation like, and I go to a foreign city, like they need to work eight hours because I'm going to want to wear them all eight hours. Like I think that's really wrong headed, just like getting navigational help that might be really useful for about 12 seconds and then you're like, great, I've got it. Or same with virtual try on like how many hours did you spend trying on glasses?

Yeah, minutes for sure.

Right, Right. So I, I think a lot of these things could be delightful but you know, less Than a minute.

Totally. Well, someone who's dedicated to their career, to this, from museum experiences to now, literally, like you mentioned, seeing the history before your eyes of a place, it's got to feel pretty fulfilling in that regard. Any last kind of piece of what you're really excited about of like, what's next in this area?

Well, I guess I love, love, love working with other people who are domain experts at energy companies, offshore wind, elevators, watches and jewelry, you know, scanning electron microscopy, you know, like, whatever it is. I feel like what's the most fun is working with people who are really like, deep in their field and they're interesting problems to solve, interesting things to visualize, interesting ways to make boaters more confident or safer. So I do feel like it's a privilege to kind of flit from healthcare to consumer electronics to, you know, all the domains that I get to work in. But the introducing people to new tools, doing workshops and teaching, it's a thrill to be able to engage with people across so many domains.

Well, it's great. And you highlight them really well in your new book, Supersight and kind of. And these chapters that really focus on specific areas. So excited to. For more people to read and engage with the book, which we'll have a link for in the show. Notes. One area that to be determined how we will experience in the future is tacos together. How will the AR world highlight. Because tacos are already at the top of our game. How much better could it be?

I mean, I hope there's somebody in the audience that builds like a menu AR app, because I just feel like we've got QR codes now all over restaurants. Like, why aren't people doing the. Like, this is what the Ramen bowl is going to look like. And then you swipe right for like all the things you want inside of it and then. And that's going to get people to have, you know, mouth watering, upselling, upgrading, like putting more stuff in your taco. So Baloco is a place that I'm in. I live in Boston and it's like a block away, so I know it's a chain, but I go there regularly.

Oh, we got. There's no wrong answer here. We love tacos for every time of day. Now we need to find a way to experience them. You and I being on different coasts, we got to find a way to experience your favorite taco in the Metaverse environment. So let's, let's find a way to do that.

That's a challenge. That's a challenge for Ms. Frizzle. Like we have to get small and jump into the taco in ar.

Yeah, exactly. There's some Harry Potter ways to definitely solve that as well, which more and more will become reality. So, David, thanks so much for being on the podcast. Really excited to continue to hear what's next in this environment. So thank you.

Thanks Neil. Super fun.

Thanks for tuning in to the Tacos and Tech podcast. You can find all of our episodes on Fresh Beer as well as on Twitter at Tacos and Tech Pod. Also follow Fresh brewed tech on all of your social platforms and make sure to share and give a review on Apple Podcast, Spotify or Google Play. Thanks again.

Startups get the party Startups get the party started. Startups get the party Startups get the party started.

You want tech in tacos, tacos in tech. You know we got it.

Startups get the party, party. Startups get the party started. Startups get the party. Startups get the party started. Startups get the party. Startups get the party started.

You want tech and tacos, Tacos in tech. You know we got it. Hey.

Watch.

You know we got it. You know we got it. You know we got it.