THE FUTURE OF MISSION REHEARSAL

It's day 4 of the ITSEC Simulation and Training Conference in Orlando, Florida. It is a beautiful day, about 9 AM, 9:30 AM Thursday morning, and I look forward to this final. You can see here, this is the conference center, beautiful place in central Orlando. The Vardough booth. Has, I'd say, really the highest quality head-mounted display out in the world. It's not a consumer device. These head-mounted displays are $10,000 to $20,000, but they're very high resolution and they're optimized for mixed reality. And so you can integrate the real world with the virtual world. So for example, with physical simulators like you have here You see your actual real hands in the near field, and then further out you see the virtual training. I tried one of the demos here yesterday and it was just awesome seeing my real hands, not a virtual representation of my hands, but my true hands with that video see-through. I also love what they're doing here with these Pelican cases. This is pretty damn cool from a, um, sort of hardware perspective.

Absolutely.

So these things fold up into a Pelican case A couple pieces go into a different Pelican case, but pretty darn cool. Here's an example of a dome-type display where you have the physical simulator inside. Obviously, this is a flight simulator situation here. So again, I'd say this is a form of virtual reality, definitely different than what, um, and maybe a bit larger than what a lot of, uh, A lot of people understand headsets and virtual reality to be. And so again, pros and cons of a large display like this versus the more smaller footprint head-mounted displays. These are historically what have been used for flight simulation for many, many years and really well refined and very much does— fulfills the needs given that they've had years to refine this. And so the physical controls are super essential here of how do you interact and fly these things that just doesn't make sense to do completely virtually.

This year we are displaying 2 flight simulators, the Atlas, which is a dome where we have 8 projectors showing at 4K resolution, pixel shift going 6K resolution with 120 Hz. On the other, Draco, we have 87 4K projectors showing 8K resolution in 240Hz. So this is near eye-limiting.

Very nice. Yeah, yeah. So that's one of the advantages of these systems is that high resolution over, you know, head-mounted displays. So a lot of our viewers, you know, understand virtual reality, they know the consumer space, but they have no idea, you know, old— like the stuff like this.

For the pilots, they move their head around and you will catch movements. So with, uh, with the glasses or goggles, you won't, you won't see the plane coming at you from behind.

Yeah, in your periphery.

Here you can do exactly the same thing you do in a plane.

What's your field of view here if you're sitting in the middle of that thing? Holy cow, it might be a ways before head-mounted displays catch up to that, I think.

It's so— the pictures are so good.

It's so Yeah, yeah, definitely high resolution. Maybe what are the, like, the high level— but you know, there's pros and cons to different technologies. What are, uh, some of the different benefits of using this at a high level versus other consumer stuff?

This is like a single relay. You don't have any breaks between training here and going into an update. Use most goggles, you will have to have Some time machine, you've used them until you can go in some things.

Yeah.

And this you can test realistically on this one. Yeah, you can, uh, ask the models, you can connect them, and you have the resolution. The resolution is amazing. And you can, you can add, add it to a larger screen. Now we have here, uh, this is a 1.5-meter and a 2-meter radius.

Okay.

But we deliver them in any radius.

Yeah, different options. Hey, you know, and one of the biggest benefits for these systems is you can't have these physical simulators, right? Which, uh, yeah, if you have everything's virtual, you're not getting that like tactile sensation that you can, you know, integrate in these, right?

So you can, and you can move it, put this on a full motion platform. Yeah. And you can make it even more realistic. Okay. And then the 2 pens will slide together and do tanking.

Okay, yeah, maybe we'll stop by then. And if people want to find more out— more about you guys, where should they go?

3D.

Excellent, thank you. Okay, so what we have here is Adaptive Immersion, their explosive ordnance disposal. That is things that blow up, go boom, explosives. And Philip Mongos, we actually competed with him with the Air Force, uh, Half Works Challenge Years ago, and they did a great job. They were the ones that ended up being selected, and it's evolved into what you're seeing here. Oh, I should say Philip Mangos— Philip Mangos is who we had on our podcast recently. And so here we have him in person showing what we have that I think we discussed a little bit of this during the podcast. And so whether it's hazardous materials or it's radiological sources out in the world, such as this downed aircraft that had nuclear material in it, Or it's actual explosives. VR is a really great use case for that because it's really difficult, obviously, to train with real nuclear devices. Generally not a good idea to let that radiation out into the world, not just for the user, but the environment.

Basically what you said, and I would say that we have very interesting stuff to look at, very realistic in VR.

It's a great environment to do realistic training, but without all the bad stuff that you're going to have to be exposed to in the real world.

Definitely something to do if you're looking to train dangerous jobs.

Yeah, repetition, practice. And during the podcast, we talked about, like, you know, you being a human factors guy of actual data and projections and those sort of things. Do you have— are you doing that with this specific project?

Absolutely. Any virtual reality or simulation training deployment, we really want to avail the user experience.

Here I am at the exhibit floor of the ITSAC Simulation and Training Conference in Orlando. Amazing show floor here. It's the last day of the conference, a little bit slower than other days, but you can see a lot is still going on here. Hundreds and hundreds of booths, and I say this is the world's largest virtual reality conference, even though it's not marketed as that. It's definitely larger than other conferences that are maybe in the hundreds or small thousands. This can be tens of thousands of attendees. I'd say about 50% of the booths have different XR headsets and demos that maybe I'll show here. Simulation and training really is the sweet spot of where Significant value, significant savings, significant impact can be gained from these immersive technologies, whether it's the actual headsets or it's other forms of virtual reality, such as these physical simulators that you feel like you're immersed.

Exactly.

Or walking into large rooms surrounded by screens. Those are also forms of virtual reality. So it's all about Visualization, but also how you interact with the world and different components. That is a human-machine interface. And then what are the benefits you get out of it, right? You can gain that situational awareness, for example, with augmented reality, and you can train over and over again for those really dangerous situations that you just can't do in The real world. Or if you do, maybe you only train for them once because it's so expensive, and then you don't train for the exact scenarios because you have such a short training time. With these different simulation technologies, you can train for all types of different factors, scenarios of different situations that might come up, so you can be truly prepared when you find yourself in those situations that can literally be Life or death. Wish you could be here with me. Obviously won't get the experience without physically being here. Maybe you'll have an idea of what it's like, some of the value that some of the exhibitors are showing here that are actually being utilized today. He's in this driving simulator. We have a gunner over here in the same virtual vehicle, and so they're working together. Can you go ahead and like look at your hands? I really want to show that. So you can see there, that is— oh, let me go. You can see he's seeing his real hands there integrated with the virtual world, which is freaking awesome. That's really tough to do. They do it really well here. And so you can mix that physical interaction of what you're actually doing close in the near field and combining that. So you have this sort of advantage of taking advantage of both that. So here's VRmed. They are doing law enforcement training scenarios with full-body tracking, multi-users in the same physical environment, very, very similar to what we were doing years ago with NIST and the Department of Homeland Security for de-escalation training and stuff. You're getting that behavioral awareness of what your colleagues are doing around you. So it's about not just individual training, but team training and situational awareness of what others are doing. Sometimes they have a weapon, sometimes they don't, and sometimes it's really difficult to tell. What are they taking out of their pocket or out of their jacket? Is it identification or is it a weapon? And so you have to make these split-second decisions that can be the difference between life and death. You make the wrong decision in real life, you may not see a tomorrow. And so to be able to train is very, very important. You find yourself in changing factors such as distance, To the person and also giving you feedback. For example, do you need to back up? You're too close to this person. He's a dangerous situation. How do you talk to that person as other user or an AI agent that interacts with you where you can ask them questions and such? And are they behaving suspiciously? And is it suspicious enough to actually use force? So really tough. decisions that need to be made. Also control those factors and make them identical between sessions, between users, so you can compare performance for identical situations instead of, well, this guy maybe did better because the situation was slightly different. Well, here you can control all those factors and thus have a more objective measure of Performance. So it's a more comparison, a more fair comparison between users. So here we have Virtre with the surround projection system. This is, uh, similar to a cave type of virtual reality display, other than, uh, this is like their head track here in this case. But you can track the head so you get the correct perspective here. You can also use, uh, 360 video, which wouldn't make sense to do head tracking, so you get those sort of very photo realistic sort of results. And so the idea is that you have that full field of view around you. I'm very familiar with the system. We're working with the Department of Homeland Security of prototyping alternatives to this. There's pros and cons like everything. And so head-mounted displays compared to this are very different and very different. Advantages of that. In our case, we were exploring full-body tracking, weapons, computer-generated imagery versus more passive type of experience that are more turn-based. But this certainly has advantages here, such as you walk into the room and you don't have to put on a bunch of hardware. Great for not just shoot/no-shoot sort of situations, but also de-escalation training. Where you can talk to the virtual characters and things like that. And what we found is that the trainees very much got into these things. They would interact as if it was real, and they would raise their voice. One is because they were being graded on were they saying the right things in the appropriate way. And so in these very tough situations, you have to use that very authoritative voice. When you tell someone to drop the weapon, you need to tell them like you mean it. And if you're not doing that, Then the instructor is going to get after you to do the right thing.

What you got? What do you think you got?

So yeah, I was explaining the differences here of, um, you know, head-mounted displays versus this, that there's trade-offs and it's a very different, uh, you know, pros and cons of each. Um, yeah, I've worked a lot with sort of cave-type systems, which, you know, it's very similar to this. And so yeah, a lot of, uh, people, they think, you know, the headsets are the only option. Right. And so I'm trying to like, uh, sort of educate, hey, there's different options out there and you choose the right tool for the job, depending on what you're trying to do.

So the headset is a great option. The problem in what we do is when you start adding in weapons. So in real life, if you just do the mathematical equation and I am 30 feet away from a human and I'm aiming center mass My sights are 1 degree off, I would miss that human being. That is just a mathematical equation. Right now, the tracking devices that you can put on weapon systems and stuff are running at somewhere between 10 to 15 degrees. So now I'm training people that is actually going to get other people hurt. So that's kind of number one.

Yeah.

That's how come screen-based marksmanship type of training is still Technology hasn't caught up. What I am gonna do though is I'm gonna have you dip into our headset. So we do things a little bit different.

Okay.

Instead of using polygons, we use video.

Yeah.

Wrap the skin so the character is going—

It's real.

Yep.

It's the real deal. I'd love to try it. All right, let's give this a try here. How you doing? I'll stop recording here, but maybe give you sort of a brief intro of what I'm going to be seeing here.

Yeah, so this is VXR. It's our police de-escalation trainer, primarily. We see some of these on DOD and military sites, so things like checkpoints, MPs, and things like that, where you're just talking to individuals who might be angry in a situation or that type of thing, and you're trying to talk them down. Blue Room.

This is our mixed reality medical training simulator. We can take the students, put them in any environments you can think of. Like right now I've got an ambulance loaded up, but I can load up, say, any aircraft you want. And when you're inside wearing the headset— I'll make this big so you can see it as well— what I'm looking at. So while you're in here, you'll be able to see your own hands work with your own equipment. If I take a tourniquet, I've got here, put this on myself when I'm inside the simulated ambulance. You've got whatever equipment you want to bring in here. I could bring in a person, I could do an ultrasound on a live person in the environment and still be able to see my screen, see my own hands, whatever we're working with. We're all about creating those environments that are hard to reach, hard to access, but still be able to take that Training and work with it wherever we need to.

That is so cool. Yeah, the challenge of bringing in the physical with the virtual, or is it the virtual with the physical? I don't know.

Both.

A bit of both.

And something purely virtual is you can't tie on a tourniquet. Probably a bad idea purely VR.

If you don't do it with your hands, you're not going to know how to do it when it's actually needed to be done. For example, we've got the C-130, we've got C-17s, the back of a Black Hawk. All those environments that you would— it would take time to schedule out just for training. Yeah, the room can just simulate that and you can do whatever training you need to.

I'm sure you could also bring in like other characters, whether another actual user or an AI agent or something like that.

Our base version of the Flowroom is a 2-headset.

All right, because you never really— almost never work in isolation, right? Yeah, makes sense.

Okay, well, thank you.

That is cool stuff. Here we have Safeguard, an old-school company of analog mannequins for, uh, the medical space of trauma. What we have here is the company True that has a really cool booth, but the point here is this is a hybrid of a physical simulator mixed With the virtual reality headset. I believe this is the screen of what he's seeing from the cockpit. So he has that physical sensation of not just what he's touching, but the motion platform. So much more realistic than just the headset by itself or just the physical flight simulator by itself. Okay, so here we have one of the behemoths in the simulation and training space. These guys create the best flight simulators in the world, and I have not talked with them yet. And so I'm going to go in and see what they got. Actually, let's take a look here. So these guys have obviously been doing this for a very long time in the simulation space. Well, maybe not obvious for those of you not Familiar with simulation and training. And this looks maybe like a little mini cave here where images are displayed on the wall. Is this a cave-type display?

So this is for our mask.

Oh, okay. Yes, acts like a green room essentially.

Basically a green screen.

Okay.

Yep.

So this is to do sort of mixed reality?

That is correct. Yeah, so we use mixed reality to Simulates the DCD, which is the depth control device right here. So you'll be able to see your hands, you'll be able to see the grip, and the grip that is being used on the flight deck, that will be used on the flight deck to control the aircraft.

And this is to be able to essentially a green screen to integrate them into the environment, is that correct?

This is like the virtual reality aspect of it. Everything in here is rendered in Unity game engine. So we will put them in a scenario for this particular demo that we have. It could simulate the catapult launch off aircraft carrier.

Okay.

Very dangerous situation. You probably can't do a lot of practice in the real world.

That's correct.

Yeah, ideal use cases. That's what's— that's—

And also the ability to do night capture.

I love how I was seeing those real hands there. So great example of mixed reality. That is a wrap for the HITSEC 2025 Simulation and Training Conference.

Saw some really cool tech going beyond the obvious and had some great demos. I think most importantly though is at a conference like this is the people you meet and the impact that they're having of how these technologies are applied. What is the result of doing all this great stuff that is really cool to see, but What's cool doesn't sustain itself long-term. What is that impact that we're having upon the world when you take off the headset or when you don't have that technology with you? Or maybe you are doing operations and you're able to visualize, so you have that situational awareness of out in the world that you just couldn't have otherwise. And going beyond Not just in an incremental way of traditional approaches, but new ways of doing things, not for innovation's sake, but for the ultimate effect and impact in a positive way that you're having upon the world. With that, I look forward to next year's conference and to see how the world evolves and what impact we as an industry are having.