#215 Seeing the Data Layer Through Spatial Computing with Cathy Hackl and Irena Cronin

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible.

It just works.

Power your data and AI transformation journey on datacamp.com.

I think we're going to continue to see not only gaming, but big Hollywood IP embrace spatial computing very early on, but changing I think the way people engage that human computer interaction in the sense that we've been really passive recipients or passive users of technology in some ways. Us adapting to technology I think technology is going to start adapting to the physical world. So that's where these new content formats, whether it's the mail as a game in spatial computing or what Marvel's going to bring us, is going to start to make us a little bit more of an active part of what is happening in these stories.

Welcome to Data Framed, this is Ritchie Spatial computing is all about the interactions between the physical world and software. With the release of Apple's Vision Pro headsets, it feels like the technology is becoming ready to use.

The tricky part seems to be that.

Many people aren't quite sure exactly what.

To use it for.

So today we're going to dig into some proposed use cases, both for individuals and for businesses. Since spatial computing has a lot of underlying data and AI components like computer vision and sensor data, I'd also like to get into how it works and the skills you need to make use of it.

Finally, having cameras and sensors everywhere throws.

Up some ethical conundrums, and it's worth us exploring what those issues are teaching.

Us about this new technology are the.

Authors of a book named Spatial Computing. Cathy Haddle is the founder of Spatial Dynamics and a globally recognized futurist. She's advised brands like Nike, Walmart and Louis Vuitton on their emerging tech initiatives. Kathy is on the Vogue Business 100 Innovators list, was one of Ad Age's leading women in 2023, and was on the Forbes LATAM 100 Most Powerful Women list in 2023. She regularly appear 60 Minutes, CNN, CNBC, the Economist, and Bloomberg. Irina Cronin is the co founder and SVP of Product at Daedos Technology, the founder and CEO at Infinite Retina and a fellow at Realigned Ventures. She was previously CEO at Transformation Group.

And Sprout Reality and has written several.

Other books around business and technology. So between them, they know more about spatial computing than almost anyone, and I'm keen to hear their insights.

Hi, Irina and Kathy. Welcome to the show. Glad to have you both here.

Thank you. Thank you.

Just to kick off, like, what, at a high level, is spatial computing? Is it the same thing as augmented reality or is it something different?

Yeah, it is a little bit different in terms of theory. So basically, spatial computing is comprised of computer vision and stuff like augmented reality and virtual reality, and it basically denotes anything and everything that could move through space that's digital or has that kind of capability. So that also includes robots.

There's a pretty broad definition then. So you mentioned, like that computer vision and augmented reality and robots. So plenty to talk about today. That's good news.

I would say, Richie. The way we've been kind of talking about is an evolving 3D centric form of computing that at its core uses computer vision, AI, extended reality, and other technologies to blend virtual elements or virtual experiences into someone's experience of the physical world. But it also means kind of a change in human to human communications and human computer interaction. That's kind of exactly what Irene is referencing. It includes robots. It includes all these virtual beings and robots navigating the world alongside us. So, yeah, it's way more than just augmented reality.

Does sound very cool. Lots of impact there. All right, so why are we here now? What has changed recently that has made spatial computing a viable thing?

I would say it's always been viable. I mean, I've been talking about spatial computing since 2018, before the term was really broadly used at all. And also I wrote a book in 2020 which used the words spatial computing in it. But what's really bringing this all along right now is that Apple came out with the Apple Vision Pro, and Apple is using the terms spatial computing versus metaverse versus XR, Mr. Arkansas VR, that kind of thing. So it's kind of the tipping point now in really bringing this technology and understanding what it is forward.

It does seem like whenever Apple gets into the game, it means something's about to go mainstream. So, yeah, hopefully this is a different point for spatial computing.

I mean, I think that everything Apple does strategic from a marketing perspective, they don't step into anything lightly. So the fact that they chose spatial computing as a term and that they describe the Apple vision process, spatial computer, I think, is very powerful. And just like Irina, I've been in the space for quite a while, actually worked at Magic Leap, which in my perspective, is one of the original spatial computing companies. We used the term spatial computing before most people were. But I definitely agree. It's a really interesting moment that we arrive at with the term. And I think for many people listening to the podcast, it might have been the first time they heard spatial computing. When Apple launched the Vision Pro and ushered in the era of spatial computing, even though it's already been in the works and told the world, this is a spatial computer.

So it does definitely sound like we're at the cusp of something interesting. Okay, so I suppose gaming has maybe been hyped as one of the big use cases, so we'll perhaps cover that first. And when I think of spatial computing games, I'm thinking fancier Pokemon Go. But this is probably just scratching the surface. So can you tell me what would spatial computing bring to gaming?

Well, a lot. Basically, the gaming that has existed up until now has used a lot of the time spatial computing, because it uses computer vision, and you need computer vision to do spatial computing. So computer vision allows technology to be able to see in front of it and be able to gauge what's in front of it, the environment, and if you're moving, where are you moving? And that kind of thing. So Niantic has always used spatial computing. There are other games that, let's say meta, that use less spatial computing. So it's a matter of degree. And that's why it's really interesting that this term is now becoming. Hopefully it's going to be broadly used, but in actuality, the technology has always been there, so it's nothing really new in terms of gaming. The new thing that might blow everything apart is generative AI, which allows people to create their own images and videos. And if you pair that along with spatial computing and also make a game with that, that really makes it fantastic.

Okay, so I think computer vision was kind of, okay, maybe like mid-2010s, and now it's getting the point where you can reliably do object detection on an awful lot of things. Yeah, yeah. It just seemed like that technology is maturing.

Yeah.

Richie. I think it's about creating that spatial awareness that Irina is mentioning, and AI now understanding the physics of the space and the physical world. So these digital elements have spatial awareness. Right. I think that that's kind of where it starts to become really interesting. And from a gaming perspective, if you look at everything that's happening in that Space. For example, very recently, Resolution Games launched demeo, which is really going to be a big title coming to the Apple Vision Pro. So I'm hoping to see how more people start to game in the device. And granted it's a device that not that many people have, but as a proud owner of one, I'm looking forward to doing some more gaming with it. And DeMeo is one of those titles that is kind of the first to arrive, that is truly in the kind of spatial computing gaming space.

So it sounds like maybe the gaming industry has a bit of work to do. Just figure out what sort of games do you need to make in this medium.

Yeah. And also, for example, Disney and Marvel on May 30 are going to be launching kind of new immersive experiences and new storytelling formats for their Marvel universe. So I think we're going to continue to see not only gaming, but big Hollywood IP embrace spatial computing very early on, but changing, I think the way people engage that human computer interaction in the sense that we've been really passive recipients of oppressive users of technology, in some ways, us adapting to technology, I think technology is going to start adapting to the physical world. So that's where these new content formats, whether it's the mail as a game and spatial computing or what Marvel's going to bring us, is going to start to make us a little bit more of an active part of what is happening in these stories.

Okay, that's really interesting, the Marvel angle. So does this mean we're going to expect new forms of TV shows and other entertainment media?

I think so. I think it's giving way to spatial video throws three centric depth content, like beyond, beyond just 3D TV. Right. I want people to understand we're not talking about like the old type of 3D TV where you wore these cookie glasses. It's more. Once this virtual content is spatially aware, you can start to engage with it in different ways because it's going to understand the physical world around you. So I definitely think we're going to see new content types. Spatial video. Definitely. From everything I'm hearing, people are getting more and more interested in spatial video, which is video that has depth. So right now, with my iPhone 15 Pro Max, I can shoot spatial video video that has depth. The video that I can see in 3D, that feels a lot more. More volumetric. It feels more like what I live in a daily basis. So I can shoot it with my phone, but then I can watch it back in my Vision Pro. So you're going to Start to think, you know, sports, entertainment, all these sorts of new industries embracing these new formats to start to experiment with what might come next. I don't know if you want to add something regarding kind of like the depth element of video and what's coming.

Well, Apple has really played up the entertainment angle and it makes a lot of sense. They've got Apple TV plus to be able to put on their own shows. So I'm looking forward to see what they produce and I know it's going to be quite a bit right now they do have some sports on there. They have soccer and basketball and stuff like that. So they really wanted to play up the sports angle in 3D and they tried to get the NFL to join but they weren't able to do the contract unfortunately. So yeah, I just want to add that there are new forms that people might think would be needed for spatial computing, but I think that the Apple Vision Pro is just pushing forward in better quality what has come before.

Certainly as someone who spends way too much of my life just staring at screens, it'd be interesting to have things go back from 2D to 3D again. Yeah, that's really interesting. I certainly imagine how something like a horror genre where you got things moving at you that's going to be even more terrifying. Let's move on from entertainment talking about proper work stuff. So it feels like any of the industries where you've got physical objects or things like manufacturing or logistics, spatial computing ought to be viable there. Can you just talk me through some of these use cases where in industries where you have got physical objects?

Well, you did mention in the write up manufacturing. Manufacturing actually has used augmented reality since around 2008. It's just better now than it was before. The resolution is better, the capabilities of texting and speaking commands and stuff like that and visuality with computer vision. So along with manufacturing, there's logistics that goes hand in hand with that. It's a no brainer that it would be really helpful not only for creating stuff, making things, but fixing machines and also teaching people how to do things. So with the Apple Vision Pro, like augmented reality, it overlays the information and visuals onto what's existing in front of you in your environment. So you can imagine how helpful that would be. I mean there's a ton of different use cases. I had mentioned that spatial computing includes robots. So robots need to use spatial computing to be able to move through space. Anything and everything that exists now in terms of computers that exist and will exist in the future will be Using spatial computing. So it's a really huge deal. And you could also use robots in manufacturing and you could use them at home for consumers. There's a whole range of other industries. Spatial computing for training goes along the whole gamut in retail industries, teaching people how to do certain things. In education, teaching students how to learn better, basically, and in fintech, teaching people and traders, et cetera, how things are moving. An interesting use case is in the Apple Vision Pro. You can have as many windows up as possible. If you want to put up 25 windows, it allows you to do that, but you probably wouldn't. But it's certainly more than what a trader would have in front of them, which would be like four to eight screens, basically, and you can move them around and do what you want with them. So that's a really impressive use case for spatial computing.

I always have way too many browser tabs open I can't see and can then find again. So, yeah, having lots of different screens open at once, if there's a better way of managing them, then that sounds like it might be pretty useful. So lots of use cases there. I don't know. Kathy, have you got any more use cases that you're particularly excited about?

Yeah, I think that there's one that I usually mention from our friend Shelley Peterson, who used to be at Lockheed Martin, now she's at Microsoft, and it was with the Microsoft HoloLens, which was an early spatial computer where Shelley's team at Lockheed Martin used the device to. To build the Orion capsule for Artemis. And I think the data they got back was really impressive. I think it was like 90% reduction in labor hours, in labor cost, which, you know, 90% is a pretty amazing amount of data. So that's one of those use cases I don't think a lot of people talk about, but very, very, very important when they're talking about how to use spatial computing in the manufacturing side. And obviously it's a very specific use case because it's one capsule for something like Artemis, but still very, very relevant. And I think it's probably one of the reasons Shelly ended up going to Microsoft afterwards from Lockheed. But yeah, I think it's a great use case on how it can be used for labor and to augment the workforce, really in some ways. But Irina mentioned is the worker can now see all this data overlaid over them. This data layer that we can't really tap into unless we have a device.

When you're trying to learn something or when you're trying to diagnose something, you've got some sort of information overlaid on top of the real world. I'm sure I've seen this before in movies. This feels like the Terminator view. This is going back to like the 1980s movies. So I guess the concept has been around for a while.

Yeah, I prefer Jarvis, to be honest. Like, this is a discussion I have very often is I rather not think Terminator. I rather think like Marvel Jarvis. You know, when Iron Man's wear his glasses, Jarvis overlays all this data on top. I think that's a better example. But yeah, I mean, okay, yeah, maybe.

JavaScript is the happy example. Compared to Terminator, I'm not as dystopic. All right, since you mentioned robots, now I'm imagining like those Boston Dynamics, like dancing robots and things. Is this the kind of thing that you have in mind for spatial computing or are there other sort of robot use cases here?

Oh, there's a lot of different types of robots, but yeah, it includes those robots for sure. You can have like humanoid robots that have two legs and two arms and the head might be there, but it might not be as descriptive as what we have in human nature. But those have a use case or many, many use cases, from being workers because they can walk around, to being in the home, washing your dishes and doing your laundry and stuff like that. Or delivering food. Walking and delivering food to you at your door. But there are all kinds of other types of robots. They're robots that are cobots that work next to workers currently that kind of aid the workers. So they're not taking the place of the workers, they're helping them. And they're usually pieces, they're like arms or whatever. They're not whole humanoid beings. Also there are these little robots that do deliver food that maybe you've seen. They're in specific locations that roll down the street where if you enter a code, you open the top and you get your food. So those aren't humanoid robots, but they do have the capability of moving through space, like spatial computing. And then there are robots that have been experimented that there's an experimentation with Amazon and certain other companies that have stopped because it's a little bit too expensive currently to try to experiment with them. But they're these robots that leave a car and are able to go up steps and are able to open doors at the same time. So I'm sure those will come back. In addition, drones are considered robots. So there's lots of experimentation done. Now Walmart is doing it. And a Couple of other companies and delivery for any kind of objects that are too large to be able to bring it down to the person and the person pick it up. There's a ton of use cases for robotics.

So it certainly sounds like any organization where they've got like a big logistics component to their business, they're going to need to start thinking about this if they haven't already.

Yeah.

There's also one more use case that I just thought of. You know how Boston Dynamics has that dog looking robot or the spot thing?

Yeah, yeah.

Okay. So there are other companies that have robots like that too. And depending on how large they are, there could be smaller ones. They could go into dangerous areas and dismantle bombs or go into areas and find survivors of earthquakes. So that's another use case as well.

So places where you don't want to send a human. Actually, on that note, I did see this company was selling, it was like the dog sort of robot, but it had a flamethrower attached to it and it's supposed to be for like dealing with wildfires. But that also sounds like a dystopian nightmare as well. Okay, so while we're talking about use cases, because a lot of our listings work in data, what are the sort of use cases for spatial computing in the field of working with data?

I'm actually working with a company right now to be very transparent. It's called Dallas Technology, and that's exactly what we're doing for the Apple Vision Pro. We're creating data visualizations and ways to be able to portray data using spatial computing. So obviously I'm a big believer in this. And I think the display of data not only in columns and rows, but also in charting and other kinds of things, if you apply AI to it, is very exciting.

That's very cool. I have a maybe nerdy question about this, but for a long time the idea of 3D plots has been frowned upon because it's very difficult to determine Anything useful from 3D plots because you have weird perspective issues. Does spatial computing solve this? Are 3D plots going to become useful or cool again?

Yeah. The reason why before it was kind of looked down upon was because the resolution wasn't good and you weren't able to zoom in very well in the 3D plot. And you can do all of that right now and you can manipulate it and turn it whichever way you want. So it's really fabulous.

Okay, so once you can spin it around things, you can get the right perspective. Answer your question?

Yeah.

Okay.

Yeah.

All right. I'm going to have to update all my data visualization advice now. I think that's very cool that that's changing.

And I think another important part, and I agree with Irene on, like, when you're in the Vision Pro and you're in spatial computing, you can actually see the 3D plot in 3D, right. It's a totally different experience than a flat surface trying to see something in 3D, which is just seems unnatural, even though we use it all the time. So that's one thing. I also think from a data perspective, what spatial computing in some ways allows us to do through computer vision and through new hardware especially, is see that data layer that we can't really see with our eyes, right? Our phones, our computers, everything's giving off data in ones and zeros. We don't see it right because we're humans, we don't consume it that way. But what you're going to start to see with these devices is that you're going to be to see that data layer, the data layer that the robots in the autonomous vehicles and the drones are using. We're going to be able to see that in annotations or holograms, all sorts of things. So I do think it opens up the access to that new data layer that we have not been able to see as humans for a long time. I even use the term like it's where we're going to meet the machine, because we're all going to be able to see the same thing. They're going to see it differently than we do, but we kind of are going to be able to see that data layer that we haven't been able to navigate through before.

Okay. So just better representations of the data are going to make things a little bit more understandable. That seems incredibly useful.

Yeah.

Plus, when you think about right now, if you're going to look at, let's say, that data layer that I mentioned, you're probably going to look through it through a phone or through a computer. Like, this is very small, Right. But when you're talking about actual devices that have a bigger field of view and everything, it starts to open up that aperture quite a bit. I always joke and I say, I hate when I go to like a Beyonce concert or a Taylor Swift concert, whatever concert you want to go to, and everyone's living through their phones, they're watching this concert, but through the phone, which is a very small kind of device in reality. But once we have this newer hardware, you're going to be experiencing things a Little bit more present. And I had that experience recently when I was on safari. I left my phone many times at the villa and I brought my Ray Ban Meta glasses, like the multimodal smart glasses that they put out. And I was actually being present and taking images of the animals as I was seeing them, but I was being present. And that to me was really transformative in the sense that I wasn't living through this little rectangle. I was wearing the device and everything, taking pictures, but I was present. I was able to kind of engage with what I was seeing in the physical world in a new way. And. And I think from a data perspective, that's something we're going to start to see.

Definitely. I have to say, all those people at concerts where they're holding their phone the whole time, very, very annoying. So if that goes away, then that's going to be a brilliant development society. Beyond that sort of data visualization aspect, are there other ways do you think that spatial computing is going to change this sort of technical work? Working with data, working with code, all that sort of stuff?

A very interesting thing, and what some people actually don't like is that when you're in the Vision Pro, it collects a lot of data on you, on the person that has the headset on. However, Apple is really into privacy, so everything will be on the local network versus going to the cloud. At least they say there's some things that would have to go to the cloud. Let's say if you're using an LLM and there's a very difficult question or whatever, and it has to do compute power, it goes to the cloud. But then it would be anonymized. So that's kind of interesting that the data that it actually gets that is really amazing is it could see where you're looking. Exactly. It's very, very precise. And based on where you're gazing, where you're looking, that's how you actually choose the app that you want to go into or whatever else that you want to choose what to do. And then obviously other kinds of biometric understanding of your face and et cetera, that kind of thing. It picks up on your own data, which makes a big difference for corporations, I think, if they're allowed to get that data.

Yeah, I can certainly see how that sort of things like eye tracking and being able to understand a lot about your personal state, that could be very powerful. So I don't know, I guess you see something horrendous. It's going to pick up on it. It's Going to be able to give you some feedback, but from a corporate point of view. Yeah. I can also imagine that's going to.

Cause some privacy issues in Irena's other book called the Infinite Retina. It's a fantastic read, by the way. But these devices scan your retina and by looking into someone's eyes, you actually know what they're going to do before they actually do it. So there's that level of intent, which from a biometric standpoint can be quite scary. But I could imagine, you know, this is an example I use often. Let's say you're on the airplane and you're going to use the free WI fi and you got to watch the ad. No one watches the ad.

Right.

You're just like, okay, you look away and then 20 seconds is over and you got your free WI fi. But with these devices, they know where you're looking, they know if you're paying attention. So, like, you can't get away from not looking at something because they will know you're looking at it or not.

Right.

So there is that level of, let's say, of intrusiveness of sorts. From a biometric standpoint. I don't know if there's anything else you want to add from the retina scanning side.

No, not from that perspective, but I did want to add from another perspective, if you attach an LLM or you use an LLM or any kind of other AI besides also computer vision, when you're looking at something, and let's say everybody knows, if you ask what kind of plant this is and then it gives you back an answer. Right. Well, this is going to go way beyond that. It's going to be able to see the panorama of where you're standing in the environment, and it's going to be able to take in what you're looking at. And if you ask an element question about your environment, it'll be able to answer you. So in that way, data is at your fingertips.

Okay, that sounds cool. Although now I'm wondering who gets access to this data then if you're in a work context. So is this gonna be a case of your boss knows your biometrics all the time? If it's like some kind of corporate headset, like it feels like there is some more stuff. Okay, so that's certainly possible. Like, they know when I'm not concentrating at your computer.

There's a lot of companies that do that now through a computer that if you're not looking at your computer, you're not typing away, you're not working and people get fired over that. So I'm sure that that'll be something that people wouldn't want, but that corporations will use. Absolutely. I mean, think about logistics. At Amazon, you gotta be working your tail off. They know when you take breaks, it's down to less detail of what that person does every day.

There's this sort of dystopian, like real optimization of worker productivity possibilities. But are there any, I don't want to completely dystopian episodes. Are there any like good ways in which spatial computing is going to help with productivity?

Well, like I said, if you look in front of your environment, it could tell you what's there. So if I use logistics again now, on the positive side, it can tell you to a degree. It could read tiny, tiny little numbers and it could tell you if the package that you're looking for is 150ft away or something like that, which is like supervision. So in that way it's a great positive. And anything where your environment is important for your job, like on an oil rig or any kind of thing where you're outside and looking at something and even inside when you're looking for something, it's a very positive thing. Also, as I said, for manufacturing and also for teaching capabilities, it can see what's in front of you and it overlays how to fix something or overlays how to actually work something. So it's a teaching computer that didn't exist before and now it has wonderful resolution and wonderful color and it's really easy to use and that's what Apple brings.

Okay, so I like the idea of teaching computer that's going to support you through things and just help you find out stuff that you didn't know before, actually. So I remember when Google Glass first came out, one of the sort of use cases they were pitching for this was that it could recognize other people and remind you of like their names and things like that. I don't know whether it worked at the time. Is this something that is now viable, being able to see someone and the technology tell you who this person is? Because I meet a lot of people and I'm not sure whether I know this person or not. So this seems incredibly useful for me at least.

Yeah, I'd say that it's so much better than it used to be. As long as people opt in to it, you'd be able to. And if there's an app for it. Currently there's no app that exists for the Apple Vision Pro. As far as I know that does that, but I can imagine that if you allow and others allow you to do that, that it would be super easy.

We talked a bit about increasing worker productivity. I guess the other maybe use case for this would be something to do with improving customer experience. Are there any ways in which spatial computing can be used to make things better for your customers rather than just directly for your workers?

For your customers? It's an interesting question. I guess if you're on the floor of a retail shop and you're looking for something and the customer is looking for that object and they can't find it, it can help you find it. So again, it's kind of like a location device that works for the customer, unless the customer actually puts the headset on. I wouldn't say that it's exactly for the customer. It's for the worker helping the customer and doing their job.

That's interesting that this is mostly about locating things and it's kind of quite close to the logistics aspect. So you mentioned a customer putting a headset on. Can you imagine any situations where that might be a good thing? Are you going to go to some store and put a headset on and show you something?

Sure, there could be several. I thought about entertainment. So it's not even a store, but like if you have some kind of movie or some kind of whatever has come out, you can have an AR experience where to overlay something on top of something else. So that's not a customer experience. But yeah, there's been a lot of tries at this and it hasn't really worked very well. Let's say you want to see how a blouse fits you without putting it on. You'd be able to do that with an app and it wouldn't be expensive. Right now it's really expensive to do something like that. They have these big obtrusive screens where you do that and it doesn't work very well. So I could see that happening very easily.

Another good example there, Richie, that I'm starting to hear a lot from the fashion industry is the stylists that work with celebrities and normally they have to send them photos or videos of some of the outfits that they're choosing for them. If they're able to do that in spatial computing, where it has depth and the star or starlet or whoever can actually see it with a little bit more volume, they might get a better idea of what it looks like. So definitely starting to see the fashion industry really get interested in figuring how to use visual computing.

That's pretty interesting. Use case celebrities trying on their clothing. I'd love to get into some of the technical details of how this works. You mentioned some of the techniques, like there's computer vision and some of the spatial aspects of things. So what are the tools and techniques that are involved under the hood?

Oh, it ranges all the machine learning and AI. I mean from deep learning, reinforcement learning, the whole gamut. It's not just one thing. I mean, it depends on the app, exactly what it uses, but it could use a combination of different machine learning AI techniques. And computer vision uses a lot of this type of thing as well. But in the guts of what it's doing for computer vision itself, that's a quick answer to your question, Kathy.

I don't really have anything else to add to that.

I would say at least with the Vision Pro, it's at least 12 cameras. There might be more, I can't remember how many cameras exactly, but at least 12 cameras which are physically scanning the world in real time on a constant basis. So that's, I think, very powerful technology that allows for that computer vision to happen. I always comment and say the Vision Pro as a device, for example, definitely has $3,500 worth of technology in it. It doesn't have $3,500 worth of value to the regular consumer yet.

Right.

Because it's not a mass market product just yet, but eventually it will be. But yeah, the technology that's in the device is extremely powerful. Those cameras and then also all the computer vision and AI that's happening in the device is pretty impressive.

What's interesting is Apple doesn't like to use the word AI and they will be using it very shortly for something else for LLMs. But when it comes to hardware, they've shied away from ever using the term AI and they always use machine learning, which is very interesting. They want to make sure that people aren't afraid of using their hardware because anytime you say AI up until recently it was sometimes scary for some people.

Yeah, we're going back to term. That's interesting that machine learning is considered a better term here because like quite often I spend a lot of time talking about machine learning and we have to put AI in the marketing stuff because then more people listen to it because AI is cool right now. So it's interesting that Apple have gone in the other direction. Let's talk about implementing this in business. We said it's not really a consumer thing as much at the moment. It's more of a sort of business proposition. For organizations who want to get started with spatial computing, what is step one?

I think anyone listening to this needs to understand that. We're not telling you, go out and buy a Vision Pro today. But one of the ways to start thinking about what this means is when computing starts to understand the physical world, how does that change your products and services? So starting to think about that. Reading our book is a first easy step to start to think about how you start to implement this. I think we're going to start to see, like Irina mentioned, Apple bringing more announcements and more things that might force people to really rethink what spatial computing is and try to take it more seriously. Those are some of the things I would say kind of try to understand the shift that's happening here. But by no means are we saying, go buy a hundred Apple Vision Pros. I don't think that that's what we're alluding to. Irina.

Yeah. Recently Tim Cook said that 50% of the S&P 500 companies have bought Apple Vision Pros and they're currently evaluating them. So that's interesting. If he didn't say that, people really wouldn't have a clue. That's a lot of companies. That's a lot of power. So what are they doing with that? Well, they're onboarding some people to figure out what the power of the Apple Vision Pro is, what it could bring, what the use cases for the particular companies, how easy it is to use, which it's super easy to use, and what are the limits of its use. So the whole range of how does this piece of hardware work and how does software get integrated in it? And they might even be thinking about, well, what kind of software would we build for this thing? For a use case, it sounds like.

Most companies are in very much a sort of prototyping stage at the moment. I'm curious as to who needs to be involved in this. So which teams or roles should be thinking about? Well, do we need to adopt spatial computing or not?

Let's start with it and innovation. So there's a lot of people in major corporations that are ahead of innovation. So it would basically come from there. It can even come from higher up, the C Suite. And then they tell the innovation person, hey, we really want to do this. You have to now have a plan and figure out how to get this used. And then the head of innovation goes to it and then it goes down from there. So basically the power spot for this would be the innovation person.

And I've definitely had really great conversations with like CIOs, CTOs, even CMOs that have asked their teams to look into this. So definitely there seems to be interest from the C suite, but definitely head of innovation down to the tech teams.

So it sounds like from an executive point of view it's going to be the Chief Technology Officer who ends up as head of spatial computing as well. Or are there other executives that need to start worrying about this?

Yes, like she said, the CMOs, so the chief marketing officers. Yeah, but even CEOs might say depends on how innovative the CEO is.

All right. So yeah, I guess it depends on the level of innovation within your organization and how sort of forward thinking you need to be. I guess most of the applications of this are only going to be as powerful as the data that's put into them. So what sort of data should organizations be collecting in order to be able to make use of spatial computing in the future?

Now I'm thinking of combining it with an LLM again. So if you put in all kinds of data and lots of data which you could carry into spatial computing headset, and then you apply an LLM to it to crunch the data, I'd say there's no limit to what can be done with data these days without the LLM. I would have said, sure, there is a limit and we don't know what that is, but there's a limit to it and it's going to be much slower. But right now, considering that you have LLM capability with spatial, it's just amazing. I think.

Okay, lots of possibilities. I guess the tricky part is going to be trying to figure out which project to start off with and some of the stuff like you suggested. Okay, let's have a fleet of delivery robots. It sounds like quite an expensive first project. Is there anything simple that you think a lot of organizations might be able to get started with? There's like a first spatial computer project.

Yeah. I'd say if you go back to logistics and for logistics, I wouldn't say the people that are on the floor exactly. I'd say the person in charge. So it'd be management because you don't want to outfit the people on the floor with $3,500 headsets, even $2,000 headsets. Right now, the people on the floor could do very well with $300 headsets and do their job. So we're talking more about management capability and using the headset, they would be able to locate where workers are, they would be able to locate where pallets are and what's going on with their scheduling and all that kind of stuff, that's not a hard thing to build. They already have stuff like that, but it's on a flatty surface versus a 3D capability. Once you put in 3D on there, and also being able to do things with your hands, to be able to control things with your fingers and your eyes and everything, it becomes so much easier and so much faster to do your job.

When you've got expensive technology, you probably want to think about management use cases before everybody use cases. That makes sense.

But there's also another one that it's being used in right now that you might say is not that simple. But there are a lot of surgeons right now that are using the Apple Vision Pro, both for prepping. They have people that are prepping for the surgeries, taking a look at visuals and then helping the surgeon go through those visuals and also overlay during surgery. There were surgeons that were using the Microsoft HoloLens to do this before, but the resolution on the HoloLens was just not good at all. And in terms of mistakes that can happen, that's a very expensive mistake that you don't want to make. I think that this will be used very, very quickly in a whole bunch of different surgeries. And they already have software to do it. It's just they have to implement it into the Apple Vision Pro.

I can certainly say the cost of getting a surgery wrong is just incredibly high. Certainly roasted for the cost of an Apple headset. That seems like a great use case. Kathy, do you have any examples of good getting started use cases?

Well, I definitely want to talk a little bit about the medical side. Like Irina was at an event I spoke recently. We did a live demo of the Apple Vision Pro on stage and I chose an app called Animarez which shows you a 3D model of a beating heart both in a healthy state and a non healthy state. And when we were doing that, the fact that I could manipulate the heart, make it bigger, make it smaller, I could go into the heart. There were so many things I could do with it that I could hear the crowd going, ooh, ah. Like so. It definitely, from a perspective of the medical professionals, is extremely powerful. I also think from the perspective of the patient can be extremely powerful because I'm going to be able to meet with my doctor and better understand what is this procedure I'm going to have, what is it going to do to my body, how do I better educate myself before the procedure? Right. Because all of us tend to go and Go down the web and then, you know, we might get all these crazy ideas. But if we actually have this 3D model and I'm in a virtual space with my doctor understanding what this procedure could do or what is going to be happening, I think that's a very powerful tool, both from an education perspective for the medical professionals, but also for patients. So I do want to say that I find that an interesting use case for sure.

Yeah, definitely. I mean, certainly there's been a lot of occasions where you go to the doctor and they give you a load of jargon about what's going on with your body and then you have to go and look it up. And it's a time consuming process just being able to actually see what happens inside yourself. That I'm sure that can be like a powerful educational tool, but also for lifestyle change. Stuff like you want to see what your liver actually looks like, maybe you go easy on the drinking after that.

Well, with technology nowadays, you can create virtual twins. You can start creating virtual twins of someone with like the MRI scanning, everything that's happening. You can start creating virtual twins of people. I think that there's extreme power in that, in how you start to look inside and educate yourself about what's going to happen in a procedure or something.

So, okay, I'd like to talk a little bit about business risks and regulations and things. So are there any regulations around spatial computing and particularly the ones that affect businesses?

I actually know quite a few lawyers at Perkins Coey.

I've.

I brought up that question especially having to do with privacy. And right now there are no real specific laws. Like if you're talking about eye gaze, there's no law that talks about eye gaze or regulation that talks about what the limits are there. It's basically the corporations that have to come up with their own procedures and their own regulations and then have the buy in from their own customers. People who are using their products to understand how far it goes. But I believe in the future, yeah, once this stuff is around long enough and people who aren't even using it have a need to be able to regulate it, there's going to be very precise kinds of regulation about it. People especially that haven't used it could be quite afraid of what it offers. If you're using it, you know exactly what it entails. But if you're not, it's hard to tell how far it goes.

I also want to add the state of Colorado, for example, just passed a law protecting people's brainwaves. So that's A president in some ways. Right. It's just a state law, et cetera. But I think that that might come into play when you have these neural interfaces that are tapping into intention. So I think that that's interesting. And from a regulatory standpoint, I live in Washington D.C. and I'm having really interesting conversations around virtual air rights, like who owns the air around me? Because with spatial computing, what you have is the physical world becomes a canvas for the device. It becomes canvas, but at the same time it becomes real estate. So therein lies that issue of like who owns the space around me and who can actually advertise there. What are they gonna be able to put in front of me? Because yeah, with some of these devices, what you're gonna start to see is that everything within eyesight and earshot of you becomes a canvas, but also real estate. So that in itself ushers in a lot of questions around privacy, like Irina's mentioning.

That sort of reminds me of the early days of the Internet where there were just banner pop up adverts everywhere. And it feels like if we repeat that with spatial computing and there's adverts flowing, floating in front of my gaze all the time, that's going to be horrendous. I'm hoping no one's seriously considering that. But Kathy, is anything happening in that area then?

I don't think anything's really happening with virtual air rights just yet. It's really early and I don't think lawmakers are quite wrapping their heads around that. There's so much more they're wrapping their heads around right now, but we'll have to see. I mean, we have a whole section in the book where we talk about some of the considerations. So I definitely recommend as a thought experiment to definitely tap into that part of the book and think about what does this mean for the future if you're a lawmaker, for your future constituents.

All those sorts of things. I do imagine that in the future, in the very near future though, there will be advertisements that pop up in front of you, but you would have to opt into that. You know, it's not going to just show up and then you're going to have to opt out. At least that's my hope that that would happen. And I'm pretty sure that that's the way it's going to go.

Okay, that's slightly reassuring. Just on this sort of privacy note, basically most personal data are protected. Any kind of medical data about yourself is protected. I'm not sure about things like is your location does that count as personal data and things like you talked about it can track the state of your retinas that feels like it ought to be medical information that's protected, but I have no idea. Are there any sort of limits to what is protected and what isn't?

There's certain regulations that are talking about retina scans that are in place right now, but they're not laws yet, so it's still in motion. Location doesn't necessarily have to be anything too bad as long as it's anonymized. So you don't know who's at that location, but you know there's somebody there, so that could be perfectly fine. And in fact, if you ask Apple this, they'll say that all of the information that is available is anonymized so that they don't know unless you have an app that you've allowed to be able to go into your data and you'd have to opt into that. And Apple allows apps to be able to do that. They wouldn't know who it was. But yeah, you can imagine Apple uses retinascan. So it's like that's gotta be really carefully under lock and key that no one gets access to that for sure.

Absolutely. I guess I don't want all that kind of my retina scans being made public. That's sounds weird and a bit intrusive. Let's talk a bit about how spatial computing is going to change people's jobs. What skills do you need in order to take advantage of spatial computing?

It depends on what perspective you're talking. If you're talking about someone who uses a spatial computer, that could go the gamut from no education whatsoever, because the Apple Vision Pro is extremely easy to use. And in fact, they've done obviously lots of testing about it and they found that almost anyone can use an Apple Vision Pro. So from a user perspective, it could go from really easy to quite difficult depending on if you have a special app on there and you have to learn what that app is for the corporation. Now, if you're a builder of software, I'd say it's just along the same lines of someone who knows CGI or has been building for iOS because the operating system for an Apple Vision Pro is very close to an iOS operating system, which is your phone. So actually people who are building for the company I'm working at, Dattos technology, have built iOS stuff for iOS devices and they've had to learn SwiftUI and a couple of other things that Apple uses, but they're very easily building the app right now. So that's not even hard either. The things that could be a little difficult are since the Apple Vision Pro operating system is quite new, there are things that aren't that obvious that you need to ask someone like a developer relations person. I really can't figure this out. There's no documentation. Can you help me? And in fact, we do have a developer relations person and are able to answer to ask these questions that are answered by them. So I'd say it's really, if you're a developer, it's really not that difficult to just decide to now do the Apple Vision Pro.

And is it important to have skills in like spatial data analysis or spatial statistics? Do you think that's going to help people adopt spatial computing?

I'd say those are completely new areas. Spatial data is something that is uncharted. It's like we're building now this whole thing about spatial data. Yes, there are companies like virtualytics that have done spatial data, but they do it in a different way. And there's some other companies, but it's not out there in the populace that, hey, there's this thing, it's called spatial data, right? How do I learn what to do with it? So I think that the Apple Vision Pro is going to democratize this to an extent that it'll be really easy to figure out how to use spatial data. It'll be really easy to integrate the data into something and then have something else come out of it that is spatial. But right now it's a totally new.

Field just to wrap up. So we've talked a lot about the Apple platform. It feels like the Vision Pro is the V1 viable platform. What are we going to see in the future? How are things going to change? Are there going to be competitors? What's going to come next? Kathy?

I think a lot of people agree that we're going to see a Cambrian explosion of new hardware over the next six to 18 months, especially with everyone trying to put AI into everything. And some of these devices are going to be spatial computers.

Right.

So we're going to continue to see Meta advancing in some of the things that they're working on. Apple as well, Samsung. So I think we're going to continue to see a lot of new hardware types for sure, and definitely new software.

Okay, so competition coming soon. That's good to know. Competition is good for the consumer. And Irina, what's in your crystal ball? What are you forecasting is going to happen soon?

It's been leaked out by Several people that Apple is building two different new Apple Vision Pros. One that is more simplified and less expensive, to the tune of something like $1,500. And another one, which is the second version of the Apple Vision Pro that's currently out there in terms of timing when this is going to come out. Originally it was supposed to be next fall, and now they're saying that it's sometime in 2026 or something like that. I can't really add any more to that, even if I knew more that I do actually know. But yeah, so I'd say that it's really great. We do need a cheaper version. That $3,500 price tag is way too much for a lot of people and is not going to be able to make it easy for most people to buy, nor would they want to buy it at that price. Other things Apple is working on Apple glasses. So sometime in 2027, 28, they're going to come out with glasses. They've been working on glasses forever. I wouldn't say that it would eat into the Apple Vision Pro because they'll be for very different things. With the glasses, you can go running, you could play sports, you could do all kinds of things, and it'll have a different use case. Even if the Apple Vision Pro becomes lighter right now, it's a bit too heavy. You certainly don't want to go outside, although people have done that for pranking or whatever. Go outside and wear it and walk through traffic with it. But with the glasses, you'll be able to do a lot of different things. But as Kathy said, there's a whole bunch of other companies that are coming along that have been working on both headsets and glasses, and we're going to see both those variations coming within the next three years.

It sounds like we can have a sort of distinction then between the consumer space and the business space.

For spatial computing, I'd say that the Apple Vision Pro with the cheaper model at around $1,500, can very easily become a consumer thing.

And then do you have any final advice for any individuals or organizations who want to get started with spatial computing?

I'd say that it's super easy to just jump in and do it. Besides the price tag, just buy the headset and use it and see how actually it works and how you don't need to teach anyone how to use it, and that there are lots of use cases for it. And it's just the beginning, so don't be afraid to jump in and try it. Okay.

I like it. Just go and jump in. And, Cathy, what's your final advice?

Yeah, I agree with Irina. Jump in, test it out, and definitely read our book. I think that'll help you kind of get a more grounded understanding of what spatial computing is and the role AI plays in it.

All right, super. A bit of homework then, for the audiences. Just jump in and try this thing out for themselves. All right, wonderful. Thank you, Irena. Thank you, Kathy. It's been great having you both on the show.

Thank you.

Thank you.