She built a VR stronghold from scratch | Annie Eaton x Jason Jerald

This is Impactful Realities. This podcast is for those at the front line of innovation who thrive in chaos and engineer breakthroughs. We're talking high-pressure industries, high-stakes decisions. We go deep with leaders designing AI-driven virtual and augmented realities and building cutting-edge solutions for the world's most dangerous, demanding, and high-stakes environments. If you're ready to uncover what it really takes to create new realities that defy limits and reshape industries, Then listen carefully as if the future depends on it, because it does. Here is your host, Dr. Jason Jerald.

Today, we have an amazing guest, Annie Eaton, who I met a couple weeks ago in Houston at the Industrial Immersive Conference. And she gave just a really spectacular talk about impact and what that means for business and ROI. And so, I just immediately saw she would be a perfect fit for this podcast. Annie has been in the XR industry for a little bit over 10 years now. In fact, she's written in her book I have here, The Extended Reality Blueprint. Highly recommend. I'm about maybe a third of the way through it, still getting some great insight from it. But it really sort of is about my understanding. I guess, Annie, you may be able to tell our audience better than I do, but those that are looking to get into XR as far as offering value such as training or how the technology can be used. And so again, I highly recommend that book. Some of the examples that she talks a little bit about in that book, but what we'll be talking about in the podcast is what are the real numbers, right? We can all sort of talk about how awesome this technology is and how cool, but we're beyond that. The technology's matured more than that. And so she actually has numbers in place of, you know, what are the response from their users? And how is it affecting business? And so, for example, she's worked with Mars Wrigley and Delta, where she's helped them save over $20 million combined savings through XR training. Those are pretty impressive numbers, and that's only across 2 companies. And so just imagine what this could do for your company.

Well, thank you.

So, Annie, why don't we go ahead and start off and kind of tell us about your story and your journey of where you came from And your motivation to move into the VR side of training?

Yeah, I do not have a traditional, like, software developer or game developer path when it comes to getting into VR. Like so many people I know, it really happened because someone gave me the VR bug. And I'm not talking about a computer bug. I'm talking about just the desire to explore this amazing technology. And that happened for me back in 2013, 2014 when A coworker at a former company we both worked at brought in the Oculus DK1 from the Kickstarter and was showing it around to people at the office. And all it really takes is that one evangelist. His name was Peter. And bringing that in, showing people in person, I think showing people the technology is so important to getting them excited about it. And that is exactly what happened for me. And so, It kind of evolved from there in this interest initially trying to get it sold into that company. I was in the marketing department at that time and I thought, oh, what a great way to use this is for marketing. Unfortunately, it was a little ahead of its time for me and my role there. So it was a no, but it got me thinking about how else this technology could be used. And so that same friend and I started a meetup group in Atlanta. Now called XR Atlanta, now has over 1,600 members. But that community and that enthusiasm that all of these individuals had for the tech really was my motivation for doing what I ended up doing. So we, out of that meetup, ended up starting a company because there were so many people asking how they could use the technology. And again, marketing background, I'm thinking that type of use case. So, a lot of our early projects were product visualization and event marketing solutions and that type of work. And then, as time went on, of course, those projects are really fun. We even got to work on one for the Super Bowl. You can't really get to a bigger event than that. But as we kept going, training kind of emerged as this use case that was provable. And I think that's what, you know, a lot of our conversation's going to be about today is What are the best use cases for VR and what can they, what can the technology do to help organizations achieve that return on investment? And so even though, you know, my heart is with marketing and these are really fun projects, I started thinking about the future of the technology and where my business was going to thrive and training was the clear path. And so that's kind of when that shift happened around 2021, 2022. Um, it, it was kind of an instant shift into, oh, this makes so much sense. We're already leveraging the work we've done and the, the skills we've built, but applying it in a way that is provable and really showing these incredible results to these organizations.

Yeah. And so in some way my path was very different, but in some ways similar. And so, yeah, I remember back in the days, not DK1, but DK2, I literally like, so like my eyes were watering cuz I was so close to crying and I was trying to be a tough guy and not cry, but it was tough. It was just compared to, you know, the different headsets I used up to that point, this just was, it was so good. And so, yeah, I could tell that was going to be a big change and also the meetup path. So I ended up taking on the meetup, the VR meetup, the RTP VR meetup, Research Triangle Park in North Carolina. And it was crazy. The first meetup was, you know, maybe 5 people. And then I think it was pretty close after the DK1 launched, you know, suddenly 300, 400 people are showing up at meetups. And all I did was like post something. It's like the most amazing.

Yeah.

Yeah, marketing you could do, but that's not what this podcast is about because back then it was a cool thing and it still is a cool thing, but people are starting to realize you need more than that cool factor. You know, what is the real business case? And that's what I really appreciated about your talk at the Industrial Immersive Conference is, okay, what is the real result and what is the real impact and what is that implication for business beyond just, you know, all of us being blown away by this technology that maybe, you know, fades after we've seen a lot of great stuff?

Yeah, I think, I think one of the biggest things that you have to do when you're starting is to find a really good use case. I mean, that sounds like it should be so obvious and it's not, you know, that hard to do once you really start thinking about it. But something that people wanna do, as you just mentioned, it's cool technology, it's still cool. It was cool back then, it's cool now, but we're trying to go beyond the cool and we're trying to come up with something that's cool and effective. And so, coming up with that use case and really thinking through what results you're trying to achieve and why it's important is such an integral step to do at the very beginning of any project. And one way that we do this, defining the use case, is to Look at different costs or even losses within an organization and identify those. And then think about what— this is going to be a double negative, but like what loss can be reduced or what can be gained by either helping people perform their jobs faster or more efficiently or with less machine downtime. And when we can come up with like where these gaps are, where this loss is, and then find a way to mitigate that through VR training, I think that's where we've found the most effective use. use cases. So, I would say, yeah, think about what you want to gain, but think about also what kind of loss you want to prevent.

Yeah, and I remember that's one thing I like too. I made it that far in your book so far as I continue reading through it. But yeah, not necessarily being attached to, you know, the first use case you see. And so, yeah, what we like to do, and I know other organizations like to do, is when you go into a company, sometimes they have a really good, you know, fit for the technology and they sort of already figured it out. But oftentimes, they, you know, aren't familiar with the technology, where it might best fit. And so it, you are new to this kind of keeping an open mind of where it might fit within your organization, where you can get the most results. And part of that too is not just where the biggest, you know, where the money is, right? Which is certainly a factor, but is it a fit, right? Because VR, XR, it's good for some things and not for other things.

Yes.

And so you need both of those of, you know, the impact, but then actually, is it feasible? And is it feasible with, you know, today's technology versus 10 years from now?

Yeah, no, that's, that's a great point. And I think just thinking about things, especially when it's training related, that are either impossible to do in the physical world without proper training, or things that are dangerous, or things that waste material, or things that cause, cause a lot of wear and tear, those are really good use cases as well. So thinking through—

Yeah.

why, why you want to use VR in the first place is big because, you know, it's very different than e-learning. It's very different than PowerPoint. But some things you don't need to replicate in VR, and that's okay.

Yeah. And, you know, really the best ideal use cases is doing the impossible, right? The things that you can't do with traditional training, that's where you get the real savings or, you know, ROI. Now, those are tougher to find. So those, you know, yeah, aren't going to happen all the time, but that can be the ideal. But there's certainly other, you know, smaller sort of things versus the impossible that you can Yeah. And so, yeah, maybe you could talk further about actually talking with the, you know, the experts or maybe the, you know, someone that's, you know, if it's an assembly line sort of thing, it's not just talking to the decision makers, but, you know, figuring out where the real barriers and, you know, challenges they're having, those that really know their stuff, the subject matter experts.

Yes. I, I definitely think that speaking with the subject matter experts as early as possible, even before the project or even before the use case has been approved. approved is really important. And it's funny, I actually had a great call with a subject matter expert for a project we're thinking about yesterday, and she didn't really know too much about VR. I explained, of course, like what we were looking for, and I asked her what part of the machine process that we're trying to train for, like what is the most fun thing to do if you know what you're doing? Because we were trying to come up with a really good first use case, very small demo. And in a lot of cases, a way we'll kind of grab attention is thinking about something that's going to be really engaging and interactive in VR, because it's like we're trying to catch people's attention. We're trying to give them something that's going to benefit them, but also do something that people are going to enjoy learning from. And so we worked together to come up with this, and she's like thinking about it. She's like, I've never really been asked to think about what part of my job is the most fun. But then she was able to define a really important part of the process, something happens once an hour, so they have to do it very frequently, and something that they're trying to get their time down to where very experienced operators can do this process in a matter of 1 to 2 minutes, but newer employees might take longer. And so by just thinking about what was the most interactive piece of her work and reframing it as like, because you're an expert at this now, what is fun to you? She was able to help me come up with the best use case for VR. So that we could potentially bring this project to fruition. So, I mean, some of the subject matter expert conversations are fun, but then we have to like draw it out of them because they don't necessarily have exposure to VR and all of its capabilities and also its limitations. So I think educating the subject matter experts is also a really important part of that process. And then once the project starts, the most fun, if we're going to continue on with the fun thing for my team to do, is a lot of our clients let us go to site and do the work ourselves. So we get trained, we learn, we get to do things on the job. I know we're going to be talking about delta deicing a little later, but they actually let one of our designers go up in a deice bucket and actually deice a plane with water in the summer. And it gave him, of course, the most realistic experience because he's doing it for real. But then that helps us be able to not only design the best way to do it in VR, the most realistic way to do it in VR, at least to the limits of VR. And then also testing. So when we're going through the testing process and making sure that this is going to be effective, because he had had that hands-on experience with the subject matter experts, that only helped us be able to verify our own work. So there are all sorts of ways that we work with the subject matter experts, but I think it's integral to include them in the entire process from use case definition All the way through testing and deployment.

Yeah, that's interesting. Sort of what we noticed is talking with the SMEs, you know, the actual users, is they wouldn't come up with any of these ideas themselves, and we wouldn't come up with the ideas ourselves. So, an example I'm thinking of is air monitoring. So, you know, are there toxic gases in the environment that you need to measure for? And the challenge is the trainer doesn't know You know, other than watching them, they don't have a history of where they held that air monitor. And so it's very simple. We just have the traces of where they were, and then you can visualize that, you know, in different creative ways. But we probably would've never came up with that idea ourselves unless we, you know, were talking with them and they didn't know what was possible because they didn't understand VR. And so, you know, what seemingly this kind of really simple, you know, implementation actually, you know, people wouldn't think about it until, You know, in hindsight. And so, yeah, it's so important to work with them, not for them, but actually, you know, ideally be there with them in the trenches.

Right. Yeah. That example you just brought up is a perfect way to bring the impossible into VR because you said they're not able to measure that. They don't have a wearable. And so you're giving them something that they never could have before, which is so cool. Yeah.

One of the challenges is we could not actually do that because to go into a toxic environment, there's no way they'd let us in. But interestingly, How they do their training, they just basically have a dead air monitor, and then the trainer reads out and, you know, made up numbers of what they might be seeing. And so, you know, we went through all that training, and one of our instructors, he said to one of our engineers, he's like, you know more than my students do. This is crazy, right? Because he got to know the problem so well and thinking about it more than just, you know, following the directions.

Okay.

But that was only because he had talked to, you know, we've been talking to a lot of those subject matter experts and really understanding the problem.

Yeah.

And not being confined by, so much the way things are done now, you know, because you're able to think outside that box when you come from an outside perspective.

Yeah, that's great.

So, um, yeah, I think maybe going into specifically with Mars Wrigley, what you did with them, and also the de-icing talking about with Delta, but starting with Mars Wrigley, what, um, sort of the overview of what you did with them and then moving on to the biggest wins.

Yeah. Well, so that project, it ended up becoming a series of simulations. So something I do like to share, especially for those who are trying to get buy-in from their organizations, is we did do a proof of concept with them first. So it actually didn't have to do with the machinery. We ended up Coming up with the primary use case, but it was a process that's very common, of course, lockout/tagout in many different industries. So, something that's very familiar and straightforward to understand. And doing that on a piece of machinery, kind of simplified for a proof of concept, but showing what could go wrong if the person did not perform that task properly. So, when we were first working with them, they really wanted to catch the attention of leadership and say, this is effective. If you do this in VR, your brain is going to get some sort of, I don't know, inspiration to do your job properly because this is like the worst thing that could happen to you if you don't do this job correctly. And it is, I mean, it's not the worst thing, but crushed hands and hand injuries are very common in manufacturing. And so that was something that we wanted to visually represent. And so in this demo proof of concept, we did show that we had a lockout/tagout process, a simplified version of it. You had to follow the instructions. If you didn't follow the instructions, your hand got crushed. And because your hands are tracked in VR or with controllers, it's something that's very easy to like attach to the physical position of your hand in the real world. And you're seeing it and you're rotating around and you're like, oh my gosh, what happened? And that aha kind of in-your-face moment really caught the attention of people within the organization to believe in VR and believe that this could be effective.

I'm curious.

Yeah, go ahead.

Are you familiar with the rubber hand illusion?

So I don't know it by that name. Is it— maybe explain it to me, or is it like the ghost phantom limb thing?

Yeah, essentially. Yeah, yeah, same area of research. So essentially you'd have your real hand, um, your right hand, then you'd have a mirror being your left hand, but it would just look like your right hand, right? But, um, And then you'd have that rubber hand and they would pound the rubber hand with a hammer or whatever. And people would like that visceral knee-jerk response. Like, it would not be good. I mean, it gets sort of, you know, bordering on ethics if you should be doing that or not, but they'd freak out as if it actually happened. And so it's this very visceral response. That's basically exactly what you're doing here.

Yeah.

People like, it's not just this like theoretical logical thing of, oh, I crushed my hand. Like, You feel it. You, I don't mean you don't feel it physically, but you feel it emotionally.

Right. But even using just very light haptics in the controller and just having that vibration occur right when the hand crushes, like that really jolted people. So, um, we put what we call like safety incidents in as many projects as people will let us. Um, we even have a new, a new safety simulation coming out this June. That's more of a licensed product. And every single one of the lessons have a safety incident that you see. Not all of them are happening directly to you. They're happening kind of in front of you, animated in VR to your coworker. But we have found so much success with multiple customers in showing what can go wrong that I am a big believer in it. And of course, doing it tastefully and ethically and not making a joke or a game out of it because—

Yeah.

Yeah. These are like real problems. They are real safety issues, highly dangerous environments sometimes. And so it's like people see these demos and they think they're funny and I'm like, yeah, it catches your attention and it might seem funny to you, but like this could actually happen and it does. And when people think about that, they're like, okay, now it's time to take it seriously. So yeah, I am a big supporter of putting that stuff into VR to an extent, of course, not traumatizing people. But really helping them understand the importance of safe work. So that was a really, really great first project with Mars Wrigley. And then again, that kind of got enthusiasm and buy-in to come up with the next. And the way that they had identified what machinery we were going to do VR training on was really identifying where there were opportunities for growth within production, also reducing wear and tear, reducing damage on the machine. And then the fact that this one site that we worked with to start had a lot of these machines, a lot of replicas, and they were doing a lot of hiring. And so it was just a really great candidate all around. And for this, I like to say there are multiple ways that you can scale VR within an organization. So, a lot of people think of scaling just as more people using it, more disbursement throughout the organization and all the different sites. But this was very different in where I would consider that scaling wide across an organization. I have kind of called this scaling deep within one very, very specific, I guess, content or machine. And so, we had produced 4 different projects for this one piece of machinery, but each of them served a different purpose, whether it was general familiarization, troubleshooting the machine when there was an issue, performing this inspection that occurs quite frequently. So performing that process and then learning about different pinch points and hazards. And so I think coming up with this very comprehensive, intensive content library for this one piece of machinery, they were able to find Astronomical results by implementing VR. So it, it started very small and then it grew. And as it, as it was more effective, they just invested more into it. And that's where we came out with this great case study.

Yeah. I can imagine the costs there are probably just incremental because you all have a lot of the assets and such already. So it's just kind of creating new training modules.

Yeah. That's a great point. We did already have, from the first machine familiarization simulation, we already had modeled the equipment and we already animated it and separated a lot of the parts to be interactive. And so, yeah, each one of them we did not have to start over. We just added on more and more detail as we went.

And I know you, you mentioned you'd been working with them from— you work with your subject matter experts or, you know, those on the line from the beginning, but then also you're getting feedback from them throughout. So what was sort of some of that early feedback or doing polling or, you know, what was the reaction?

Yeah, I mean, the reaction was really, really positive. So that's always a good sign. I think something that we did, which I would encourage others to try if you need it, is we would— we brought the 3D models and everything into an AR app and lined it up with the real machinery and just made sure everything was matching up. And we Met with the subject matter experts during that process as well, and they were giving us like kind of these little tweaks because the difference about this project than some of our others is that we did not have any 3D models to start. So the machinery is older. It's had a lot of different additions and changes that were highly custom to this company. And so it's not like this is just something that's that the 3D models were actively available for. So that, I think the subject matter experts just helping us tweak the realism of the machine throughout as we went was important. And then the trainers actually did some polling after the training classes and almost everything was very close to the top 5. I think it was like right between 1 and 5, and almost everything was an average of, I think, 4.6 and above. And these were questions like, do you think this training helped you learn? Um, was it an engaging experience? Was it comfortable? So I'm really glad that they did that because of course leadership's going to think, you know, maybe this is a great thing to do. The trainers might think it's a great thing to do, but it's not going to be good unless the trainees like it too, or at least they think they learned something from it. And so I think—

Yeah, everyone needs to appreciate it.

Yeah, everyone needs to appreciate it, especially the trainees. And, um, because they're the ones that it's ultimately impacting. And so I think hearing that feedback from them was one of the best parts of it.

Yeah, we had crazy net promoter scores, which basically asks how much would you recommend this to someone, which was pretty much perfect, which was crazy. I mean, we've done, you know, other real-life workshops, right, where, you know, pretty good results, but I've never seen results like this. It's crazy. And so part of that is the power of the VR. Now, of course, you can't just assume just because it's VR that it's going to get that. It's got to be well done and all of that. And that's not the only factor, but yeah, for engagement, it's just, if done well, it's just spectacular. But beyond that, what were some of the higher-level metrics you looked at?

Yeah, yeah, the good stuff. So this particular machinery previously took about 12 weeks to train their employees on, and that was reduced by half down to 6 weeks. So I think that's like, you know, huge. 3 months down to a month and a half is a big deal. And this is some pretty complex, big, big machinery. And so that is one. Now, we also heard that they had an 18% increase in employee efficiency for new hires right out of the gate. So not only are you cutting that training time in half, but you're also putting your employees out at a higher output level themselves. So they're operating more efficiently than their, the previous trainees and more of a traditional classroom learning style. So that was really positive. And then something that we heard but were not expecting was actual employee retention increased by 10%. That's not something we were trying to measure, I don't think, at the beginning, because, you know, usually you think, oh, you're teaching someone a process. It's like, are they faster? Are they more efficient? Are they learning it? But I believe that it was probably partially because they feel more confident in their jobs, so they're staying because they know what they're doing. And then partially maybe because of that reduction in training time being so significant that they're out on the job faster, they're engaged with their company faster, and they're staying. Because I would feel like staying in training for a few months, you might get kind of itchy after a while.

Yeah.

And so I think there were a multitude of factors that went into that 10% increase, but I'm very happy that VR had such a big impact on it.

Yeah, turnover and training of new employees, especially when it's extensive. I mean, that's one of the biggest, biggest costs companies have. So yeah, enormous potential for savings there, but not just potential. You guys actually calculated some of those, some of those savings, right?

Exactly. Yes. Yes. And yeah, they, they did a lot of calculations. And as we've expanded through the organization, It's been really cool to see that that is something they are thinking about at the beginning before they even approve a project. They're very dedicated to being able to confirm that they will prove that ROI. And so it's different because that's not always been the case. And so I think for companies who are looking into doing any sort of VR/VR training, identifying those metrics before you start is going to be very important if you want to measure them at the end. And for companies who aren't, because we've run into this case where they're like, well, we're not really tracking that, so how can we measure it? You know, if you're doing a custom simulation production, there is gonna be a timeline to actually produce the work. So, start right then. If you have identified that you're not measuring something, it's a lot easier in manufacturing because they always are measuring everything. But at any organization, if you have goals and you don't know how you're gonna measure, Because maybe you're not measuring already, it's not too late to start right then because you can get a lot of data in the production timeline that you'll then be able to measure against once it's deployed.

That's a great point about manufacturing measuring everything because unfortunately in other fields, we don't do that and we definitely need to. So yeah, much easier to work with. They're kind of expecting that. I'm curious, was Mars like forecasting or estimating? Obviously, you don't know the, you know, what the true numbers will be. But when they put together the project, did they have an idea of what type of savings or return they might be getting?

For the first one, I don't think so. We were lucky that the site that we worked with is known as an innovation site. They, I believe, have maybe a little bit of budgetary discretion to invest in new innovative things in hopes that they will find something that's going to be effective long-term. So I think because they are experimenting or exploring new technology more, the initial ROI calculation, I mean, it well exceeded what they were hoping for, I'm sure.

Yeah.

But I don't know that that was really a part of it. But I have noticed that every single project since, they have been very calculated about Making sure they're going to, to have a similar ROI or at least a very strong ROI.

And adding all those, 'cause there's savings across multiple, you know, multiple factors there. Did you guys, um, get like a final number what that savings was?

Yeah, I believe what they have shared, um, is about $19 million in about 7 months of deployment.

So, yeah, over 7 months.

Mm-hmm.

Wow. So, Yeah, those savings are probably going to extend.

Yeah. And that was a while ago. So I'm sure, I'm sure it's more. I know that there was an initial kind of jump because that was, that was the point, um, was this big round of hiring to meet demand. Um, something that they've told us was that just chewing gum, uh, demand is off the charts. So, um, as much gum as they can make, they can sell. Um, which I'm not surprised because I, I love chewing sugar-free gum and that was their specialty at the site we worked with. And so, uh, yeah, I think the fact that they had a product that people really wanted and they knew if they ramped up production, they could sell it, it was just kind of this perfect storm.

Yeah. Yeah. And engagement leads, you know, sort of the soft number, but it can be, it actually leads to those, those real numbers. Uh-huh. Uh, an example we had was working with, um, hazmat, um, operators and they go through basic training and then They can choose to go on to more advanced training to the hazmat technician level, but it's this very technical thing that, you know, firefighters, you know, aren't typically, you know, into the math and such that's required. And so, it's, you know, maybe 2% of those, um, go on to the technician level. And so, what we saw in our pilots that had shot to 20%, from 2% to 20% of that interest of saying, I actually want to, you know, pursue this further as a career and be specialized in this area. And that's their, one of their biggest challenges is recruiting. They're just not able to find the people that wanna do it. And yeah, if you can get them engaged, you know, you can have that major, major effect on people and their passions.

Yeah, definitely. I've thought about, um, and we've kind of proposed this, we haven't really had many people take us up on it, but kind of taking these training simulations that we're producing and maybe modifying or simplifying them to be like career exposure for these organizations to take to career fairs. I think I've heard of a couple of customers taking our sims to career fairs, but they haven't modified them. So they're just like the full-on training and they're like, okay, well maybe this is a good thing cuz you're really getting an idea of what you're doing on the job.

Mm-hmm.

Um, but I think that's a, a cool way to like repurpose. I know we were talking earlier about repurposing the machines, um, or the 3D models, but that could be another alternative way to repurpose the content, um, for recruiting, making kind of like a simplified version of it.

Yeah. Kind of a very different, um, target audience, but maybe it's similar to how, you know, you already had the assets, so it was just changing up the modules a little bit. It might take a minimal amount of work to, you know, really make the case for that specific target audience.

Yeah.

Okay. Well, why don't we, um, move on to Delta? So, uh, your project with them on de-icing airplanes, could you give an overview of that?

Yes, that one, uh, came to us a few years ago and The primary reason being that they do their de-ice training in July, and there's no icy planes in July, at least not here in the US, maybe in other parts of the world. But they get this training, this comprehensive training, and similar to what they let us do, spray down a plane with water sometimes to practice. And then there's a gap of several months before they actually have to de-ice a plane. And so there's a couple reasons why de-icing in VR was thought of as a good idea. One, allowing them to practice again after a gap. And then the second is to give them these proficiency checks or assessments to verify that they do know the proper procedures. They know what to spray, what not to spray. Actually, what not to spray is pretty important, sometimes more important because it can lead to damages. Um, but understanding that and being able to assess them right before they actually have to de-ice planes instead of July months ahead of time. So, um, when we first started this project, we just did one plane. Um, we did a couple of different contaminants, um, different like weather. I think we had snow and clear skies and daytime and nighttime. So a few variables just with one plane. Um, but then as that First year was successful, the company invested into expansion of different types of planes, more contaminant types, more environments, so, and other variables. And so it's really grown into a multi-year program, and we're about to start our 4th year of updates. So it's really cool to see how it's evolved over the years. And they again use it to assess their deicers to make sure they're ready for the job.

That's amazing. 4 years. So one of the challenges, to be honest, I see in this industry is it gets to the pilot stage and people think it's cool, but then, you know, it never lasts beyond a year. So that's, that's amazing. 4 years. I mean, obviously they're getting value out of it. And I know you had also mentioned, we talked earlier about the, the 50x improvement, right? Going from that 3 employees a day being able to train to 150 a day is just, Just insane. Could you talk about that a little bit? Yeah.

So that has to do specifically with the amount of people that they can, they could put through de-icing a plane. And so before VR, of course, you have to have a physical plane of which they are, you know, usually in the sky or being used with passengers. And so they wouldn't really be able to pull planes out of commission to be able to run every single person through This proficiency check. And so they would, of course, assess them in different ways. And the more experienced employees already, you know, were familiar with the process. But by bringing VR into it, instead of only being able to do 3 people per day, they're now able to do 150 people per day. And they have their employees, the people who are training and assessing the de-icers, they have VR kits. They've got tablet and 2 headsets. that they go around with in a Pelican case to the different sites, and they'll just pump through people doing the assessments. Again, like I mentioned, there are parts of the plane you're not supposed to spray, and the whole simulation is set up to where if any of those parts are sprayed, you know, it sends flags in the system. They don't pass, or they get a grade that they need to redo it. And so we're really making sure that, of course, you can visually see those things when they're doing that on the physical plane, but when the programming is there and they can't, I'm not going to say cheat the system because they're not trying to cheat, but it is very clear, yes or no, that you know how to do this and you know what to avoid and what not to avoid. So I think they've found massive success. And yeah, that 50x assessment rate is huge. I know they have also mentioned this is not— there's not an exact number, but millions of dollars in operational costs saved. So that has to do with travel, not taking the planes out, and also the de-ice fluid, which is not only a monetary expense, but also, you know, you don't wanna have that material waste that you don't need. So a lot of different factors that have gone into this. I've just heard multimillion-dollar ROI. But yeah, it's again, over years, it's been cool to see them continue to use this solution and find success in it.

Yeah, it's an interesting point about the fluid cost. My understanding is that is extremely expensive. And so you might or might not damage the plane, which is bad. That's gonna, you know, cost something.

That would cost a lot. Yeah.

But you're like guaranteed to be using that, you know, have that cost of the de-icing fluid. And so, um, yeah, it's like definitely guaranteed savings there.

Absolutely. Yeah, I know. That's like why we say in the summer we can de-ice a plane with water. But in the winter, if you do that, the water will freeze and that's totally not the point of all of this. So you can't even practice with water in the winter.

Yeah.

Yeah.

Good point. Yeah. Any other metrics or findings or sort of insight you got from that project?

Yeah, that, I think those are the main ones from that project. I know that we have made regulatory changes as well. So every year there are FAA regulatory changes to many different aviation processes and procedures. And so anytime that happens for the de-ice process, that's something that we update as well. I know that, yeah, I think those are the main ones that we have to share there. We have another project we didn't speak about, also manufacturing but safety related. If we're just talking numbers, I know that they recognized a 30% increase in knowledge retention. And again, this is all safety-related subject matter specific to manufacturing. So, it's just great to hear these companies actually measuring the success of these projects because, I mean, you and I have been at this a long time and it was not always the case.

Exactly.

Not only that people weren't measuring, but they weren't sharing it. And so, It's, it's an exciting time because I think we are all moving past this, like, just do a pilot and then it's over. And of course, you still have those clients and those companies.

Yeah.

But so many are seeing that value fairly quickly and then just investing more and expanding. So I would say, like, most of our customers now are repeat customers. Of course, still getting new ones, but the fact that they keep coming back over and over for more simulations, more content is A huge positive sign.

Yeah, certainly building up that trust and getting them to return is, yeah, what's important. And yeah, I mean, pretty much all Fortune 500 companies are exploring and piloting these technologies, but they're not willing to share. And, you know, they want to keep that competitive advantage. So that's great you're sharing. And then on the regulatory side, this is certainly, you know, further off, but on the training side, eventually, This is going to become, you know, standardized that you need to do some form of VR training because it's just, in some cases, you just aren't able to train for it. Right. And so it's the right way to do things. We had, when we were doing, again, the hazmat work, we had one of our hazmat trainers after he, you know, really understood it, he said, you, this needs to be in every firefighter department across the nation. Like, this has gotta be a standard. And so it will get there. But yeah, standards and those sorts of things don't happen overnight, but I can see this 5, 10 years out for sure. These are going to be standard trainings.

Yeah, absolutely.

We talked a little bit about, you know, can you forecast or project what the savings are going to be, but you never actually know those numbers. And then there's other things, you know, outside the numbers, just, You know, you're talking to a SME and you get a complete surprise. What were some examples of, you know, expectations, how you and Wrigley and Delta saw things going versus how it actually ended up?

Yeah, I think that in every project, we always try to prepare that things are not always going to turn out how we expect in the beginning. Of course, we want to know what we're going to make and most things we make. But there are some times where you either have aspirations that are unachievable or you have an idea mid-project that you're like, oh my gosh, where did this come from? And like, this is going to work. This will make this experience better, or this will help us teach this process better. And then we implement that. So, something that I think for anyone who's getting into this as either a developer or a Enterprise that's trying to explore this technology is like, know that things are not always going to go the way you expect. I'm trying to think of a direct example of that off the top of my head. And I think interactions are a big thing. So a lot of times we'll think we know how something is going to go. So actually the Delta De-ice project's a good example of this. So we have like an open bucket and an enclosed bucket because those are 2 different types of vehicles that a de-icer would travel in as they're going around the plane with their fluid. And we started with the open bucket. It's a lot more straightforward. It's on a semi-standardized path. But when we added the enclosed bucket, we wanted the ability for the person in the bucket to actually like control that and go closer or further away or up and down. And something that we didn't expect And this is just something because, you know, VR is VR, is that some— you're sitting down in real life in this enclosed bucket, but some people wanted to stand and some people wanted to sit just because of like how comfortable they were in VR. And so something that we had to end up adding in was the ability to grab the whole console and like shift it based on your height and your— I don't know if you were sitting or standing. And it's— there are different ways that people approach this, but something that I saw, and I can't remember which game, I think it was Cookout.

Cookout.

of times we'll find inspiration from games. But something I think it's Cookout that does it is like when you have your table in front of you, you can have this little lever and you can adjust the height of your table so that it's perfect for you. Because when we're designing in VR, everything is in 3D space. And so we can assume the natural height of a table, or we can assume the natural height of this console.

Yeah.

But depending on the person, that could change. So even if we have it set 4 feet off the ground, it might You know, be better at a different height for someone else, or it might just not feel quite right, or maybe they sit on an ergonomic pillow. Like, I don't know. And so being able to implement this, like, minor change to be able to adjust the height of the enclosed bucket console for an improvement in the user experience mid-project is something that, one, we did not expect, but after getting real feedback from our end users testing it, was a need. And so, I think you'll just find these, these things, a lot of times they are, they are feedback from the end user that end up pushing you and challenging you and then making a better experience overall.

Well, yeah, that's a huge part of design is that iteration and continuous feedback. So, that makes, yeah, it makes sense, but not something you would have thought about in advance. I'm curious, in the real world, do they have the choice to stand or sit, or is it always Sort of the same way you get, you know, whether you like it or not.

No, they have to sit. That's why it's— I think that's why it's a unique challenge because like their chair in their office did not feel like the chair in the enclosed bucket. I think that was throwing them off a little bit. And also the chair in the office is not the same height as the chair in the enclosed bucket. So I think just having that fine-tuning, I am someone who actually prefers to stand in VR. I know there are a lot of people who prefer to sit and people who prefer to smooth locomote. I cannot do it. And so I think coming up with, I guess, comfort settings and allowing people to experience VR in a way that is comfortable to them is important when we can. But I agree, like, because you're— I like your question about like, are they allowed to stand in the real world? No, they're not.

Yeah.

So that was a little bitty baby exception. But for some of our work, we do ask that. We're like, oh, can they sit while they're performing this job? They say no. Like, we would not make a seated option because they do have to train for what they're going to be doing in the real world. So that's— it just kind of depends.

Yeah, that's super interesting. Some prefer to stand because I would think you just feel safer sitting, but it's a good example. You don't know until you talk to people.

Yeah, I don't know.

So yeah, different preferences in VR versus real world. That's different. Although in the real world, I like the standing desk. sitting right now, but I normally like to stand and I've done that so much that I actually like it better now. It's not because, you know, I have to force it.

That's awesome. I know I go up and down and up and down all day long.

So I guess maybe there's not that much difference of VR versus real world.

Maybe not.

Okay. So as far as people or organizations, I should— looking to invest in VR training, not only because, you know, it's damn cool, But as far as, you know, business-wise, is this a good choice for them or what use cases they might choose? Do you have any sort of most important advice that you would tell them?

Yeah, I mean, I think as we spoke about earlier, finding where you can make that ROI provable is going to be important. Finding the either loss or the costs or whatever it may be that is going to be impacted the most significantly. Now, I know we've been talking about financial ROI. This can be calculated financially as well, but one thing that also has a big ROI impact is also safety training. And so, yes, it has a financial impact as well, but also just human impact is important and keeping your employees safe and keeping them Healthy and happy and working and not injured. Um, highly important. And so, yeah, there are going to be the monetary considerations, but there are other ways to calculate ROI. So maybe it's your safety record. Maybe that's not a direct financial impact, but it is a safety record that, um, will improve the livelihood of your employees. So there are many different ways to calculate it. I think there's probably an infinite amount of ways to calculate it. I think when you're investing this, as well as mentioned a moment ago, know that things are not always going to turn out exactly as you expect and be okay with that. This is still new to a lot of people. Those of us who are making it, yes, we've been specializing in it for a while, but we're still figuring things out. I don't think it'd be as fun if we already had everything figured out. And so—

Absolutely.

The fact that we get to experiment and sometimes We're creating an interaction for the first time ever in the existence of VR. Like, that is a really incredible thing. And so for organizations who are investing in this, whoever they dedicate as their XR champion or the person who's going to be like the main person spearheading that at the organization, give them a little bit of flexibility. Like maybe it can all be within some sort of scope, but like know that they, they may have to shift and tweak depending on how things turn out and how things That's another thing that we talk about a lot when producing VR is how things feel. And that is not always quantitative. That is more qualitative and harder to define sometimes. And so I think just educating people who want to get into this, that there are these intangible pieces to it and that's okay and that's expected, is important.

Yeah.

But I mean, overall, if you find the right use case and you produce it well, it's a no-brainer that you'll get a great ROI. I think it's just understanding the process and what to expect.

Yeah, I love that word intangible. So in fact, working with the, we work with the ROI Institute to, you know, analyze these numbers, but they're, you know, they're called the ROI Institute, but they get, it's not only about ROI, right? Impact is not always measured by ROI. It's a nice, you know, objective way of measuring things, but it's not the only thing. So, they use that same word you just used, as they say, tangibles being, you know, can it be converted to real money, but then also intangibles, right? What are those sort of non-monetary impact? And they're both extremely important, right? If you're trying to change the culture to, you know, fit the vision of the company, right? That's certainly important as well, even if it's, you know, really difficult to put actual numbers to it.

Yeah. Amazing. Intangible and tangible. I'll have to start using that more. Yeah.

I've learned so much from them. They're an amazing partner. But yeah, I've also learned a lot from you through your book and, you know, getting out there in the world, talking to people, having these interesting conversations. That's where the most learning, I think, comes from is, you know, us, you know, multiplying each other what we can be up to. And Yeah, what the future holds for us. So I'm excited as we continue to work towards that ultimate impact of what it means, not just for organizations and companies in the world, but society in general. I really do believe that the world is becoming a better place. And so I very much thank you for your time, Annie, and your insight and your numbers. I love numbers. I'm a numbers guy. And—

Well, thank you. Yeah, I was just gonna say really quickly, like, thank you for For hosting this podcast. I feel like it's just this conversation and from what I've seen you're trying to achieve with it, it is much needed and it's great to learn from each other and continue to lean on this community because a lot of people are wanting and willing to talk about it. Maybe not all of them want to share the numbers, but I think you're right. We can learn from each other. Everything is still so new in the grand scheme of things that there's Plenty of room to explore together. So thank you very much for hosting this.

Absolutely. Okay. Thank you, Annie. And thank you to our listeners.

The Impactful Reality Podcast requires more than a conversation. It takes action, research, and collective wisdom. If it made you think, we'd love to hear your insights in the comments or contact us directly to shape the future together. Don't just subscribe, pass the torch. Put it forward. Send it to the next one in your circle who manifests things that matter. The builder who dreams big and innovates towards more impactful realities.

The Jerald Report with Jason Jerald.