Flying high with AI drone racing at AlphaPilot

Bandwidth for Changelog is provided by Fastly. Learn more@fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out@rollbar.com and we're hosted on Linode Cloud Servers. Head to linode.com changelog this episode is brought to you by DigitalOcean, the simplest cloud platform out there and we're excited to share. They now offer dedicated virtual droplets and unlike standard droplets which use shared virtual CPU threads, their two performance plans general purp CPU optimized They have dedicated virtual CPU threads. This translates to higher performance and increased consistency during CPU intensive processes. So if you have build boxes, cicd, video encoding, machine learning, ad serving, game servers, databases, batch processing, data mining, application servers or active front end web servers that need to be full duty CPU all day every day, then check out Digital Ocean's dedicated virtual CPU droplets. Pricing is very competitive starting at 40 bucks a month. Learn more and get started for free with a 100 credit at.co changelog again.co changelog. Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changelog.com community follow us on Twitter we're at Practical AIFM and now onto the show.

Welcome to another episode of the Practical AI Podcast. This is Chris Benson speaking. I'm Principal AI strategist at Lockheed Martin and with me today is my co host Daniel, who is a data scientist with SIL International. How's it going today Daniel?

It's going pretty good. I'm working on not my normal amount of sleep. I've been having having some extra meetings this week with a team in India I'm working with on some dialogue stuff, some conversational AI and so it's been early mornings for this week, so I'm looking forward to sleeping in in the morning. But it's been been good, good tech. Just, just early mornings. How about yourself?

I probably have more sleep. I was a little sleep deprived earlier in the week. I think I've caught up, but I'm just kind of on cruise control here. I'm about to head over to the UK for a week of work and then a couple of weeks of family vacation, so I'm looking forward to that.

Good times. Yeah, looking forward to some shepherd's pie or jellied eels or something strictly vegan everything. Oh yeah, that's right. I don't even know why I asked that.

Strictly Vegan, my friend.

When I think of England, the food scene seems like, you know, meat and a pie is what I think of.

And I think traditionally it has been. And since you said that, I'll just say my wife was amazed. She's always grumbling, we're both vegan together. And she's always grumbling that her father is like the epitome of the meat eater. And she was just delighted, she was just gleeful when I came home a few days ago and she said that her dad had tried some vegan beef and vegan hot dogs and stuff like that and he liked it and she was just, oh, it's great. So who knows?

Well, you'll have vegetables, pies or something.

There you go. So, you know, apparently even the meat eating folks of the UK may go that way, but this isn't a vegan podcast. So I'll stop, I'll stop going down that anyway, but when I get back. So I will kind of be back into, into the world on this side of the pond early in October as I kind of coming back into, into work on the 7th. And I'm going to get to do something that's pretty fun, that's participating in a program that Lockheed Martin is, has been driving with some partners and it is called Alpha Pilot. And today with us on this podcast we have Keith Lynn who is the program manager at Lockheed Martin for the Alpha Pilot program. And when he tells you about this in a few minutes, it's going to blow your mind. It's a super, super, super cool thing and he's been working really hard on it. Keith, welcome to the show.

Hey guys, thanks for having me.

We have actually been talking about having you on for quite a long time with some of the events that we internally at Lockheed that you have orchestrated. And I've kind of come back and just said, Daniel, this cool thing and you know, at some point, you know, when it's the right time, we got to have an episode about it. And so we've both been excited.

I've been asking Chris each week about, hey, can we talk about this thing you've been talking about now?

He really has. So, so before we dive fully into Alpha Pilot, if you could just kind of tell us a bit about yourself, how you got to where you are, you know, and kind of what's the life path that led you up to this moment where you are running this amazing Alpha Pilot program.

Yeah, so I think I've been with Lockheed about 11 years now. I started out graduate in physics and mathematics at a liberal arts college here in Pennsylvania. And my first job was doing signals intelligence. And from there I. I guess my curiosity got the best of me. And Lockheed is a really good place if you like some diversity of what you're working on. So I quickly jumped to doing some research in healthcare, then in energy management systems, eventually made my way to our corporate office and got to do some international work. Spent better part of a year and a half managing R and D in the United Kingdom, some nanotechnology research down at Rice, Rice University in Texas, and eventually landed in the autonomous systems group at corporate. This is around the time when AI really started to take off. You know, a couple years back, AI was, you know, if you ask people, what do you know about artificial intelligence? They'd say, you mean like robots? So the. The term in itself has. Has really blossomed and it has much broader field of meaning now. And at Lockheed Martin, we started looking at artificial intelligence and then a subset of that, which is autonomous systems, and really trying to figure out where we can apply that across our different products and services. And that's what really led me into alpha pilot. A few years back, we. I'm sorry, a year back? It feels like a few years back already.

I know that when I saw you at the event that you probably mentioned a little while back, you were going, pardon the phrase, balls to the wall. You were really working hard. So I can believe it's felt like three years.

It's been one of the longest years for me on record, that's for sure. But I couldn't be happier. I might not always say so in the heat of it, but I think I always look back and I say, man, doing this kind of thing where it's chaos and things are going haywire, but you somehow manage to land the plane. So it was just about a year ago, I was actually sitting at a hotel next to south by Southwest Conference center, where we were doing a presentation. And I think I was actually telling my boss, like, I'm getting a little bored. I'm not really sure what I want to do next. And I feel like I need my next big thing, something that really pushes me to the limit. And then our vice president sat down and, you know, he just said, I just got this new thing, you know, just came on my plate. Lockheed wants to do a global innovation challenge, and they want to do something with autonomy and, you know, maybe drones or something like that. We're not sure, but it's got to be huge. Got to really get people's attention it's got to really show people that Lockheed cares about artificial intelligence. And now we just need some, you know, poor fool to put in front of this and figure it all out. And I mean, the timing was just perfect with what I was talking to my boss about. So I just said, hey, I'm looking for something, let me take a crack at it.

So before we even dive into Alpha Pilot, you know, just to give some background on, before we even talk about the program, you know, it has to do with, with drone racing and stuff. And I was wondering if you could kind of talk about what that is prior to Alpha Pilot. You know, what, what is drone racing and what's the interest in it? And is it, is it something that's specific to the us? Is it international? If you could talk a little bit about that and then we'll get into Alpha Pilot and describe its role in that world.

I just mentioned that Alpha Pilot involves drones and drone racing. Specifically that as a Sport is about 4 years old from what I've been told. It was first conceived of in Australia, then made its way to America and really took off and now it's really has a global audience. I know it's very big across, all across Europe actually. But what that is basically is humans piloting drones via a, what they call FPV or first person view camera. So these pilots have a really good RF signal, so they have low latency, they see what the drone sees and they are competing with each other in a traditional first to the finish type race through various types of courses.

Like real life video games?

Yeah, like a real life video game.

And how do they see?

By wearing these goggles. And I've actually tried them on, they just look like, like a pair of lab goggles or something like that, or VR headset kind of. And you put them on and you get this view from the drone and a lot of times it's kind of grainy, it's not the best. And if you at all motion sick or you're the type of person that gets motion sick, it's not the most pleasant experience.

That's what I was just thinking. So I can ride regular roller coasters all day. But I remember like we went to Universal Studios recently and we were in one of those like 3D rides where it also like shakes you around and stuff. And man, I got so sick. It's like something about that like virtual motion just really does not agree with me.

Yeah, the older I get, the more that affects me. When I was younger I could kind of do It. But yeah, yeah, after a while, your body works with you less and less in that way.

So you mentioned that there's, like, these people, they're wearing headsets, they're piloting these drones. I'm assuming it's, like, around a racetrack. Like, there's gates they have to go through or something. Is that how it works?

Yeah. So it's done a little bit differently depending on who you're racing with. There's a lot of different organizations that are doing this. Some are more grassroots, some are professional. On Alpha Pilot, we're partnered with the Drone Racing League, which is probably the biggest drone racing outfit out there. They're on national television. They have events worldwide, I think over 80 million viewers annually. Their courses are like a traditional race course where you're going through different gates and checkpoints, and they try to do them in exciting locations and integrate with the local environment to the extent possible.

So you mean, like around buildings and such?

So I was at one last year that was at the BMW headquarters. So there was literally an indoor track with all these BMW vehicles going back 100 years, and they were flying over top of the cars and between different BMW displays. So they try and make it really, really an exciting environment.

Cool. Yeah, that's pretty neat. So how fast are these drones going? Like, I'm just imagining having this headset on, and I'm assuming you have to have pretty quick reflexes to kind of make it around this course. What's the sort of speeds that these things are going and, like, the size of the course? Like, are we talking like, feet, miles? Like, how big of a scale are we talking about?

So I think even more than reaction time, it's so darn hard just to control the drone. There's so much sensitivity in the joysticks. I've tried it out with a little, you know, like, training wheels drone, and I could not. This thing just pinballed between the floor and the ceiling. So it's. It's incredibly hard just to keep the thing in the air. But they top out around 80 miles an hour, I believe. That's not sustained 80 miles an hour, but, you know, on a straightaway, they get up to about that speed. The laps themselves, they're maybe like a size of an indoor track.

I would say, like a track and field track sort of thing.

Yep. And they would do multiple laps. And the. The interesting thing is the, you know, they're really burning through these batteries, going at that speed. Speed and with that kind of agility. So the flights themselves don't Last very long, maybe a minute total.

Like the full race, or do they, like, stop for, like, pit stop, recharge? I imagine that wouldn't be too exciting.

Yeah, so when you're there, I mean, you got this safety net in front of you, There's a very intense safety protocol, and these things just go buzzing by you, and really, sometimes all you see is just the LED lights. So it's just kind of a blur of red or blue or yellow or whatever color that particular racer is.

It seems like almost like a horse racing experience where, you know, at least a couple people on this call don't approve of all things horse racing, but it's like that very short period time of, like, high excitement and, like, everything's over and however long it takes, one minute, like you said, it's a very, very intense experience.

Yeah, you're. You go from looking for it to come around the corner, whiz past you, then your eyes go back to the giant screen to see where it is now. And, you know, Drone Racing League is very good with production quality, and they have the whole arena mic'd up. So if one of these things bites it, you know, anywhere you're gonna get that audio as if it's right next to you. And, I mean, it's a real intense sound when these things crash going that fast. And part of the guilty pleasure of racing always is seeing a little bit of the hardware go flying. So it's a fun time.

Yeah, you can describe it in a minute. But even the thing that I was at, the event that I was at with the teams was. Was quite thrilling. And it wasn't even production quality of what. What you're describing for everyone. But I was, you know, I think. I think all of us that were new to it were like, wow. Which kind of brings me to the next thing is, you know, now that you've kind of described what drone racing is with these human pilots doing their. Their thing really well, could you kind of describe what Alpha Pilot is and how it fits into that context of drone racing and such?

Sure. So Alpha Pilot is a public innovation challenge. So I would compare it to like a DARPA Grand Challenge or an X Prize, where an organization, in this case Lockheed Martin, puts out a hard problem. So ours was programming a drone to fly in a race course autonomously, and that challenge is associated with some kind of award. An award. So ours is $2 million for the top teams and then an additional $250,000 kicker prize if any algorithm can fly faster than a professional human pilot. On the same course. So we're having a little friendly rivalry with the professional pilots here in the drone Racing league.

Gotcha. So there's like, it's almost like a sort of Watson Jeopardy sort of challenge thing going on where there's actually humans racing at the same time.

It would be more like if there were four different Watsons competing with each other and then we took the best Watson and put it against the best human Jeopardy player just for fun.

So there's I guess a series or a bracket or something too. So like people go against each other, the humans go against various drones, they're autonomous. And then there's sort of like a grand showdown.

So originally we were going to pair the autonomous racing with human racing so we could have more of that side by side. Then we decided to move towards more stable standalone events because we wanted this to really be not just about the racing, but a celebration of artificial intelligence and create an opportunity for larger STEM engagement. So our events are just going to be autonomous racing, even though we'll have some human pilots there to do some commentary and some demonstrations. And then as we get later in the season, we will have one of the pilots on standby to go head to head against the top autonomous team after the racing for that day is finished. This episode is brought to you by Brave. The Brave team is on a mission to fix the web by building an open source, privacy focused and performance oriented browser. Browse the web up to eight times faster than Chrome and Safari, block ads and trackers by default and reward your favorite creators with the built in basic attention token. Yes, you heard that right. A real world use case for blockchain. Download Brave for free using the link in the show notes and give tipping a try on changelog.com.

So for these teams that are working on Alpha Pilot, you know we've talked about kind of the drone racers in the manual sense. There are these teams that are behind these, these automated AI enabled drones that Alpha Pilot's doing. Could you tell us a bit about the teams and you know, where they're from and a little bit about them?

Sure. So Alpha Pilot as a innovation challenge launched late last year and we had about 430 teams sign up to compete and these teams were from all around the world. We used a simulation based process to down select to just nine teams to go into this first year season of autonomous racing. And the teams that we selected are just fantastic. A lot of them are with universities, some of them are small businesses, but we have teams from Georgia Tech, ucla, University of Zurich, team from Brazil, team from Warsaw, team from KAIST in South Korea, and then we have a team that's really kind of a mix of people from three different continents, which is quite impressive considering this is all software based and they have to run it remotely. So we're really impressed with the diversity that we received and there are some pretty impressive folks.

So on these teams, I'm kind of curious, as you mentioned universities, are there kind of mostly student teams that are kind of doing this as a, let's say a senior design project or something like that, or is it like graduate researchers or what's the sort of range of experience there?

A little bit of everything. So we didn't want to put too many restrictions on how teams formed because we wanted people to feel comfortable working with the other individuals that, you know, made the most sense and that they would enjoy, you know, being part of this with the most. And we do have some teams where it's all undergrad students, we have some where it's a mix of grad undergrad and we have some where the professors are even involved. So yeah, we get a little bit of everything. The only rules we put on it were no more than 10 individuals per team. So we didn't have anyone with 100 people on the roster. And then we have to keep tabs on all of them. And they're outsourcing tasks to, you know, the web and things like that. And then a basic age requirement just because you're dealing with some, you know, flying things that are really heavy and go fast. So we want to make sure everybody's an adult. But that was it.

And in terms of constraints, like technology wise is there and forgive me of my ignorance of drone racing out there, people probably love drones and I'm like butchering a bunch of things or ignorant, but is there like in these leagues, is there a specific drone you have to use or like what are the constraints as like hardware wise, like you have to have a drone that's this model or one of these models, but you can modify it in XYZ ways. How does that kind of work for the teams?

Yeah, it's a good question. And in the entirety of drone racing, the answer is all of the above. Some leagues will let it's byod bring your own drone and they'll just give you some basic requirements about size, weight, power. Others are a little more structured, but still byod. So they might say you have to use, you know, six inch props, you have to use these motors, but the platform and everything like that. You optimize as you want. And then the drone Racing League, our partner in Alpha Pilot and their new league that they're creating for Alpha Pilot and then will be a standalone event moving forward, is called Air Artificial Intelligence Robotic Racing. They use a standardized drone and they do this in their human races as well. So each year they. I believe each year. But periodically they release a new model drone that is standard for all of their competitors. And we really like that a lot. Key because we wanted this to be about the quality of the algorithms. And if a team won, we wanted to be because they were the best programmers, not because they had better motors or better batteries on the drone or something like that.

Yeah. So I guess to extend that a little bit, we're kind of talking about some of the tech with this. Could you kind of describe the technologies that are in the drone and you know, in terms of the hardware, what kind of hardware is being used to run the algorithms and do inference? Just kind of take wherever you think would be appropriate. What are some of the technologies that are being used at this point by DRL's drone for these races?

Yeah. So the main difference between drones for autonomous racing and drones for human piloted racing is that our drones have to carry a computer on board. And to do autonomous racing you need. There's two ways you could do it really. You could do edge computing, which is all the decisions that are made by the algorithms happen on board the drone, or you could off board, which means you have a low latency comms link between the drone and some high performance computer on the sideline somewhere. And it reads in signals, sends it to the computer. A computer makes a decision, sends it back. That's less impressive technically. And it could actually create a problem because of the latency and the speed of the drone. So we decided to do fully onboard edge computing, which means you need a pretty impressive processor on this drone and there are some drones that carry an onboard gpu. But we wanted to make sure that we were giving people enough computing horsepower. We went with the latest Nvidia model for edge computing, which is the Xavier. And that has never been put on a drone before. It's a little bit larger than previous model GPUs, so we had to design an entirely new platform altogether. DRL did a fantastic job doing heat displacement and everything for this drone. And it's definitely unique and first of its kind.

Yeah, I know. Commenting on it and I actually have the Xavier listed here in terms of its specs. This is the same GPU computer that is Used in autonomous vehicles. It's a 512 core Volta GPU with tensor cores, 8 core arm with a 64 bit CPU, 16 gig of 256 LPDDR 4x memory, 32 gigs of flash storage. It's quite a computer. Without going through the whole thing, I know that when I saw you last at the event, you're putting a pretty serious computer on these drones. And I was pretty impressed with the performance even.

So, yeah, no wonder they run out of battery. I mean, I'm kind of impressed. This isn't like, you know, the humans are still wearing the headset and they're kind of like augmenting the flight somehow. It sounds like more. It's like they're. And I don't know how this starts. Maybe you can describe it. Like there's these autonomous drones that are just kind of hovering there, ready to start. And like you push the button, start and like based on the, the video feed that they're processing, then they sort of accelerate, turn, switch direction, slow down, all of those things. Is that the kind of scenario we're dealing with or what data are they working with to make these decisions?

So the teams are primarily training for these races in a simulation environment that's built off of the DRL training simulator. So they modify that for autonomous racing, they put in the new drone physics and updated world models, and then they also created course models based on the races. And we have a permanent training facility in Littleton, Colorado where we set up a small scale course with two gates, and that's kind of the gym, so to speak. So these teams can write their algorithms and then train them like in a simulated deployment in this gym. And then what they can do is when they feel comfortable, they send the code over to our team, we load it up on an actual drone at this facility and deployed in real life. And they collect the actual data from the drone, the telemetry input from the IMU's rangefinder, the cameras, and send that back to the teams. And then the team uses that in comparison with what they're seeing in simulation to make some modifications. And then before each race, we release a model of that course so they can specifically start training for the race course that they're about to compete in. And then about a week out from the race, there's a final code submit. So they send us the code, we do all of our safety checks on it, make sure it checks out, and it's going to deploy. And then on race day, they have absolutely no physical involvement with the Drone whatsoever. Everything that it does when it takes off on that podium has already been, been coded and pre planned and it's just ready to go when they get there.

So I would like to follow up on a couple of things that you just said. And first of all, there's literally a podium that all of these drones are sitting on rather than being all the way down on the ground and they take off. So they're at a slightly elevated position just for listeners. But you mentioned a few things, a little bit of jargon there. I want to ask you for some definitions for, just for, so everybody can follow. You mentioned simulation and world models and a couple other things, but starting with those two, can you kind of talk about what is a world model? What does that mean in simulation?

Sure. So it's really, really hard to train a drone to fly itself when you're primarily doing it in a simulation, which means you recreate the world that you want it to fly in in a computer environment. And the reason that's hard is because there's always a delta between what's actual and what you're simulating, what you're modeling. So in order to do that, you need high fidelity. You need to model everything the drone's going to encounter. Wind, lighting, glare, drift in the sensors, or the standard deviation error bar. And the sensors that you're using, which is specific to the specific brand and the hardware. Gravity. You need good gravity, wind resistance, dragon. So all these things have an effect and if you don't model them well, that means your drone isn't going to behave exactly as you think it is. It's going to behave slightly differently. And sometimes that's small enough that it doesn't make a difference. But if you're going fast and over a long enough distance, that could be the difference between going through the center of a gate or crashing into the edges of it.

Gotcha. And you also mentioned a couple of other things. Just for definition, you mentioned telemetry, which probably most people know, but I'd like to cover that for a second. And also you mentioned imu, and I'm betting a lot of people don't know what IMU is referring to.

Yeah, IMU is inertial measurement unit. And this is like a. You can think of if maybe some people are more familiar with a gyroscope, something that essentially can provide data on the three dimensional positioning of a drone at any given time as it's flying. So pitch, roll and yaw are the three measures. Could be leaning forward, left or right or Side to side. Knowing your position in space is really important if you're trying to figure out how you go from where you are to where you want to be. And then also knowing your position from your last measurement is important. So you can gauge your velocity and.

Trajectory and just to kind of separate. So there's like the simulated world in which you can kind of train before they ship the code off to you, and they kind of go through the training at the gym or whatever. But in terms of the. So there's that simulated world, but in terms of the actual inputs that the AI models are working with in order to, like, make decisions, is that just like. Just video? Like, like a human operator would see, or there are other. Some of these other sensors that. And these measurements that you're talking about, can those actually be input to the. To the mod as well?

So the simulator that our teams are using is part of what they call the integrated development environment. And it's a hardware in the loop kit. So think of it as a computer that's running a simulator. Then the simulator is connected to basically a dissected drone. So there's the cameras, and this drone in particular has four cameras. There's the inertial measurement units that I mentioned. There's a laser range finder, which points down at the ground, and that gives the drone a measure of its altitude at a pretty high sample rate, something like 200 hertz. And then there's the Xavier itself. And when they're running the simulator, everything they're getting on the screen. So let's say they have an environment that looks like a warehouse and they're flying towards a racing gate. That visual input is then run through the cameras, which then output that into the gpu, which then processes it. So it's replicating the exact process that they would be using on the drone.

Yeah. So there's definitely various inputs to the AI models, but in this sort of simulated environment, those inputs are simulated as if they were flying around the track. Of course, like you mentioned, there's like these deltas that they have to deal with, and it's not always exactly the same, but it's meant to mimic that as close as possible.

That's right.

Makes sense. It seems like a heavy, like with four cameras, this laser range finder, the gpu, all of this stuff. It's like a pretty intense drone. How big is this thing and how long can it run? You mentioned the races were about like, what did you say, Like a minute or something?

Yeah, the human piloted races are about a minute long. The drone itself could probably run for about five minutes before the battery started to run out. And how long it takes really is going to depend on how proficient our teams are with their algorithms, which is, you know, partially based on their ability to program, partially based on the fidelity of the simulation environment that we're providing. And you know, we're working really hard to continuously upgrade that to make it as accurate as possible. If everything's looking great, they should be able to fly through this thing and in, you know, seconds, you know, 30 seconds, 40 seconds a minute. But if you look at the current state of the art, so to speak, you know, there's been other attempts to do autonomous drone racing. Iros, competition comes to mind. They go pretty slow there because it takes a long time to orient and figure out what they're doing. So still I don't think it would take more than two, three minutes tops.

You know, Alpha Pilot is a fantastic way of showing, you know, how current AI technologies are driving autonomy kind of in a small, a small scale and stuff, but just in a larger sense in general. How do you see AI as an enabler for autonomy? What has Alpha Pilot brought you to in terms of how you think about this since you're part of this world on a day to day basis?

Well, the work I've been doing with Alpha Pilot and with artificial intelligence in general has been pretty enlightening for me. It's something I haven't specifically studied, but the range of applications is just so broad and it's not always in the most obvious ways or the ways people would think. You know, self driving cars always come to mind, or robotic systems come to mind. But basic optimization of business processes or human functions like scheduling and calendars and email and things like that, you know, AI is already making a huge difference and I think that it can't be overlooked because that's really how it touches everybody's lives. That really gets it out there. Everybody gets some level of exposure to it. Things like home assistants work in the language space, others work in the written space. And I think the more we all become familiar with AI and the more we have opportunities to see it doing cutting edge things like flying a drone all by itself in a very friendly environment, in a competitive environment that's fun and encouraging, the more everybody is going to get inspired by it and we're going to come up with even greater ideas.

So partly I'm assuming that Lockheed is interested in this sort of innovation challenge because of the algorithm and the automation part of it. That could be applied even outside of flight, but inside of flight, like when we're thinking about maybe like commercial airline flight in the future, or even recreational planes or military planes or all of that. Like, what do you see as kind of the, the near term applications of, of AI within flight? I know that like the, for example, the car industry, they're kind of going through all, you know, figuring out all of this stuff and there's talk about, oh well, it's first going to happen in, you know, long range trucking and there's always going to be a, like a human operator there. And like, what, what does that look like in terms of flight? What, what do you kind of see as the sort of near term applications of AI in the, in flight, whether that's commercial or military or whatever.

Yeah, so it's always fun to talk about what AI could do, but I think it's equally important to talk about what it can't do, what it should do or what it shouldn't do. And when you're talking about military aircraft or space exploration or anywhere where you're putting a human in a dangerous or difficult and extreme environment, there are certain things that AI should be doing or could be doing that really improve the safety or overall quality of that mission. And those are generally lumped into things like cognitive assistance, where somebody under stress has a high cognitive load and they make mistakes, or they're more prone to make mistakes, or where their reaction time is limited or it suffers to some extent. That's where AI can really help save lives. And we are using it in that capacity already. Another area that's really interesting is teaming environments where you have a human piloting an aircraft and you may have that human commanding a fleet of autonomous vehicles. So it's really the interchange between human pilot of vehicles and unmanned or autonomous vehicles working together. And then they really become an extension of the human. They provide additional mission capacities or they help them with certain functions and basically extend the reach.

So in terms of like, if I'm thinking of commercial airlines flight, for example, it seems like if I compare the sort of drone racing you're doing, which already, like within this confined space with like a small device, we're talking about like four cameras, a laser range finder, all of all of these inputs that it's receiving and it's a challenge, would I be right in saying that at least in the, in the near term, like AI might be processing inputs like within a flight to help augment pilots, but it may be unlikely that we're going to be seeing autonomous 747s everywhere really soon.

Yeah, I think we can certainly get to that point someday. But in the near term, whenever you have precious cargo or human lives on board. This is what I was talking about, where you need to think about the limits of AI or what AI shouldn't do. And right now there is a certain need for a human to have a level of control in that decision making loop. And if you're talking about passenger aircraft, I don't think you're going to see them fully automated anytime soon. There's a certain level of comfort and dynamics that a human can bring to decision making that I think you need a pilot for. But you might be able to say, so let's say there's a pilot flying a 747. I'm outdated here. 747 is so old now.

Yeah, I was trying to think of a.

Don't make it a Max. Okay.

Just being fully transparent here. Being on a call with two people from Lockdown Martin and trying to come up with a Valid aircraft name 747 was the only thing I was really confident in. So that's what I went with. But I realize that's probably extremely out of date. I wouldn't, wouldn't want to go for like the Boeing Max, whatever.

So I think it would be totally feasible in the near future to see a pilot land their 747 and the onboard AI is flying in parallel and then that pilot can compare their performance to the AI's performance. And either the AI learns from the pilot or the pilot learns from the AI. So those types of things are already in motion and they have tremendous benefits attached to them.

I'm curious outside of, you know, we've been talking about flight, you know, within air, within the atmosphere. I'm just curious what your thoughts are in general about other environments and you know, you know, whether it be space, whether it be underwater, and you know, where, when we might see inroads there. Do you have any insight into any of that?

Yeah. So, you know, we're at Lockheed Martin, we're looking now at autonomous vehicles for use in humanitarian aid and disaster relief. And there's a lot of different scenarios, especially in that area where you have difficult to reach environments or you have destruction of local infrastructure and it's really hard to put a human in there. And it's very time sensitive. So that's a great near term application where autonomous vehicles can do a lot of good. You're already seeing things in the news about them being used to Deliver blood and medical supplies to hard to reach places. That's another immediate application. And then as we really start to grow into things like space exploration, going back to the moon or putting a human on Mars, you get in these situations where you have communications latency given to the huge distance and you need to make decisions local. So you need an autonomous vehicle that can process information and make its own decisions. So I think you'll see as we get into the new space age, you'll see AI playing a huge role there. And then I think you mentioned undersea as well. And this is an area what's actually already in practice with some of the autonomous vehicles we're developing at Lockheed Martin. We've already used them for things like undersea oil rig inspection after some of the hurricanes in the Gulf. And just yet another domain where it's hard for a human to go safely and it's also hard for a human to analyze the situation very well because it's not their traditional element. So these types of things are just right for AI.

So if I'm, and I've probably shown my ignorance of drones and flight throughout this interview, but if I'm out there, maybe our listeners are wanting to, you know, get involved with, with drones and maybe do their own experiments. Are there accessible ways that they can kind of experiment with? Maybe it's computer vision or other things with drones without kind of being part of this elite group of teams competing at this level. I think if I remember right, there's like, there's at least some drones that have AI inference chips on. Am I right about that? Maybe, maybe you know of some good ways to get involved.

So there are a couple commercial drones that are starting to have onboard computers where you can, you know, you could do some amazing stuff with them. And they're a lot of fun things. They have like an onboard Raspberry PI or Nvidia Nano. And that's enough compute for you to basically learn the basics of how to code so that a drone can read input from its cameras and use that to make guidance, navigation and control decisions and, and fly autonomously. So there's a lot of great beginner kits out there. And then if you want to experience what some of the pilots are dealing with for Alpha pilot and for the air season, we actually have the simulation environment we used in our qualifier, which is different from the one the teams are using now, but it's, you know, high caliber simulation environment, uses the robotic operating system that has been made available open source through MIT and it is called Flight Goggles. It was developed by a consultant on our program, Dr. Surtash Karaman at MIT, who's been a huge help in setting this whole thing up. And it's a great, fantastic tool. I encourage anyone to just go check it out, even if you're just flying manually. It's a lot of fun.

Sounds great. So I guess as we finish up, if you could possibly maybe tell us a little bit about the first couple of races that are coming up, where and how people can access them. But also, as you do that, as people engage on this and families and kids are watching, what would you like to see them take away? What kind of impact would you like to have on them as they engage on these initial few races?

Yeah, I'll start with that one. This is something that I find really fascinating. I'm so thrilled to be doing racing with AI, because racing has a history of being a catalyst for getting people inspired with new technology. And, I mean, I gave a talk at MIT a few weeks ago where I was showing photos of London in the 1890s, and I showed an article in the London Times about how they were predicting in several years the city would be buried in horse manure because the current solution of horses and buggies was just not sustainable. And people were really freaked out about this at the time, and they thought, we're at the edge. This is as far as society can go. Right. And the automobile was invented and people just kind of wave it off and they're like, that thing's a monstrosity. I don't know how to use it. You know, everybody, every. Horses are so integral to society, and you just want to get rid of them and you want to use these things now. I don't know how to use it. I don't know how to work with it. And it just floundered for quite a few years. And because nobody was demanding it, the scale of production was low, the prices were high. And I think that when people started racing cars and made a spectacle of it is when really people started to come around to the idea of these things being a technology of the future. And it took kind of the mysticism out of it. And if you can watch. If you can pay a dollar and go watch these things race, then they're not something that's reserved for a select few. They're part of your life now. And that's what we want to do with Alpha Pilot. We want to make artificial intelligence and autonomy part of everybody's lives. Everybody can watch these races on NBC Sports or Twitter. And feel some ownership in it. Or they can come to our race events which are free to the public and feel a little bit of ownership in that AI and feel like maybe this isn't something that is closed off to them. Maybe it's a lot easier to get involved than they initially thought. And if they're inspired to go home and do a little more research or maybe try and see what other great things are going on in AI, then I feel like we've done our job to help inspire that next generation.

Fantastic. And when are those first couple of races and where are they so people can show up for them or watch them online?

So the first race is going to be October 8th at the University of Central Florida and it's going to be an all day event. We're going to have some great Lockheed Martin products on display. We're going to have the UCF Drone Racing team there showcasing some of their capabilities. Some of the DRL professional pilots will be there doing the same and we'll have some great commentary about what people are seeing in the races and about AI in general. So tickets will be made available on Ticketmaster and you can find that through the DRL website, the DroneRacingLeague IO or DRL IO. And yeah, the races will also be included in the Drone Racing League's content on NBC Sports and on Twitter. And we are filming all of this, all the behind the scenes, what the teams are going through, what we're going through, what happens at the races, all this great human drama associated with the AI is going to be put together throughout the season and then hopefully released as a documentary on one of the great On Demand platforms.

That sounds fantastic. I'm personally very excited. My small role on the team is to do some AI commentary and so I can't wait to be in Orlando on the 8th of October and looking forward to seeing you there. Keith, thank you so much for coming on Practical AI and kind of taking us into a deep dive into Alpha pilot. And thanks so much.

Of course. This is a lot of fun. Thanks guys.

All right, thank you for tuning into this episode of Practically I. If you enjoyed this show, do us a favor, go on itunes, give us a rating, go in your podcast app and favorite it. If you are on Twitter or social network, share a link with a friend, whatever you gotta do, share the show with a friend if you enjoyed it. And bandwidth for Changelog is provided by Fastly. Learn more@fastly.com and we catch our errors before our users do here at Changelog because of Rollbar. Check them out@rollbar.com changelog and we're hosted on Linode cloud servers and@linode.com changelog check them out. Support this show this episode is hosted by Daniel Whitenack and Chris Benson. The music is by Breakmaster Cylinder and you can find more shows just like this@changelog.com when you go there, Pop in your email address. Get our weekly email keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.