Oliver Cameron (CEO, Voyage) - MIT Self-Driving Cars

All right, welcome back to 6S094 Deep Learning for Self Driving Cars. Today we have Oliver Cameron. He's the co founder and the CEO of Voyage. Before that he was the lead of the Udacity self driving car program that made ID as an autonomous vehicle research and development accessible to the entire world. He has a passion for the topic and a genuine open nature that makes him one of my favorite people in general and, and one of my favorite people working in this space. And I think thousands of people agree with that. So please give Oliver a warm welcome. Thank you very much, Lex, and thank you all for having me here today. Super excited to speak all about Voyage, but in reality the kind of thing I want to share today is kind of like this title says how to start a self Driving car startup. Rarely do you kind of get an inside scoop of how a startup is formed. You kind of hear all the pr, all the kind of very lovey dovey press releases out there. I want to share kind of the insight of how at least Voyage came to be, which was a little unconventional compared to your average self driving car startup. They always tell you that the path to a startup getting to the goal you want is kind of a zigzag. Ours was kind of an insane zigzag. So we'll go through all of that stuff. Stuff. Talk about my background. Also a little unconventional. I'm not very good at learning in a classroom. For me, I found learning by doing, by building, has always been the thing that's worked best for me. So going all the way back to when I was a teenager, software just in general was my passion. This idea that you can make something out of absolutely nothing and then all of a sudden millions, in Facebook's case, billions of people can be using that thing. And after building lots of crazy stuff and perhaps not being too popular in high school, because that's all I did, I started a company. I won't bore you with all the details, but learned a lot during that experience and went through Y Combinator, which I believe started right here in Cambridge, which is very cool. And then this very pivotal moment happened to me. I heard about this online class which was generating a whole bunch of scandal and lots of controversy. And it was from this guy called Sebastian Thrun. He'd taken this Stanford class he taught in artificial intelligence and just said, screw it, we're going to put the whole thing online. And back then, and this was around 2011, this was a very controversial thing to do today. MIT and many others do this all the time. But back then there was a hell of a lot of controversy around doing something like this. But this learning format really just appealed to me. Being able to sit in front of my laptop, learn at my own pace. Build, build, build was something that really resonated with me. And I took this class in 2013, Artificial Intelligence for Robotics. And this again was just this pivotal moment. My head exploded. All the enthusiasm I'd had for software kind of transferred to artificial intelligence and robotics and just became addicted to the format of what are now called MOOCs massively courses. And I loved them so much that I decided, hey, I want to go do this and help others learn this stuff. So hey, let's go join Udacity and build more classes like this. So I did that for four years, led our machine learning, robotics and eventually our self driving car curriculum, which was a lot of fun. And I got to learn directly from two great company builders, like truly great company builders. One was Vishal Makhijani. He was the operator extraordinaire at Udacity, understood how to build a company, how to build a culture, how to incentivize, and how to do all those things that we don't often talk about. And Sebastian Thrun, he of course founded the Google self driving car project in its early days. And right now I believe he's building flying cars just in general. I learned so much from him. But this idea that you are literally in control of your destiny, you can build absolutely anything if you put your mind to it, was always pretty inspirational. Today of course I build self driving cars at Voyage and we'll talk more about what makes us special compared to the other self driving car companies you may have heard of in this class and beyond. Let's talk about Udacity. Can you raise your hand if you've heard of Udacity? Very curious. There you go. That's most of the room. Udacity, like I said, was founded by Sebastian Thrun. He took this class online and all just exploded and he built a company around it. Udacity's real focus is on increasing the world's gdp. This idea that talent is everywhere, that it isn't now just constrained to the best schools in the world. That because of this proliferation of content, there are talented students all over the world and all they need is the content in which to be able to build crazy cool world changing things. And what I see as my job today is to go out into the world and find these ridiculously talented people and then put them to work on the hardest Problems that exist. And Udacity to me felt like the perfect place to do this as a kind of prelude to this. About three years into Udacity we had had this real focus, like I said, on machine learning and robotics, but we really want to take it to the next step. And we came up with this kind of concept internally that we called only at Udacity. What if we taught the things that other places weren't teaching? What if people all around the world could come learn from what may appear to be niche topics, but were just being taught at the right time because that industry is about to blow up. And the first one we did of this, and we've done some after, including flying cars, a much more in depth curriculum on artificial intelligence was self driving cars. So this is a quick video that introduces it. And this is of course Sebastian Thrun, robotics legend. Let's see if this plays the lane. Self driving car has been the dream of my life and I've dedicated more than 10 years to it. If you can build a safer car, it will save a lot of lives, but on top of it, it's transformation. Just imagine instead of owning a car, you have a little phone and you stomach the car and an empty car comes to you, jump in and lets you go somewhere. And when you don't need any work, it disappears. There is an enormous market for self driving car engineers. Lots and lots of companies that you wouldn't suspect have entered that field and are massively high. I challenge everybody here to be part of this. We're launching university in another degree program called self driving cars so that everyone in the world can become a self driving car engineer. And why did we want to do this? What was our goal? It was to accelerate the deployment of self driving cars. Like Sebastian says in that video, there's a number of reasons why self driving cars are transformational. And at the time this was around 2016, it felt like self driving cars were just taking a little bit too long. We rewind to that particular spot in time. Google was really the only main effort going on. And what we believed is that it needed to happen faster and that one of the reasons it wasn't happening fast enough is because there wasn't enough talent in the space. So what we decided to do is like I said, build something quite special. We wanted to pair up a world class curriculum, an actual self driving car, which we'll talk about more and what we called our open source challenges and all of that would come together to build this quite special curriculum. So let's Start with the curriculum. One of our beliefs was that partnering with industry was the right way to go. That was because it felt, and I believe this, that the knowledge of how to build a self driving car was not necessarily trapped in academia, it was trapped in industry. So we had to go straight to industry, work with engineers that were already challenging themselves with these problems and get them on camera, have them teach the concepts that they know and build day in, day out and have that be transplanted to thousands of minds around the world. So these are just some of those partners. There was many, many more. But we had a real focus on finding these engineers wherever they may be and getting those folks on camera. We also built an incredibly talented team. This is just a small snippet of the curriculum team, but of course Sebastian Thrun was a big part of this curriculum. When I told folks that I'd gotten a chance to work with him on specifically self driving cars, he likened it to getting basketball lessons from Michael Jordan, which I thought was pretty fun and they were probably just as entertaining. But some really, truly great folks working on this curriculum and still doing that to this day who deserve all of the credit, frankly. Here's a quick photo of our first lecture recordings with eventual Voyage co founders Eric and Mac. Eric, who's on the left, hates this picture. And here's. There you go. He still isn't at max height, but he still has that box on his desk. And we built a whole 12 month curriculum to take an intermediate software engineer who may be in consumer software or just some other part of the software world and take them into self driving cars. We wanted to cover perception, prediction, planning, localization, controls, even just the whole breadth of of the industry. And the reason we want to do that is because we saw the best fit for a Udacity student. Not necessarily being a specialist in a niche, for example, just perception. Although there's been a whole bunch of folks doing that as well. But that the skills of a Udacity student tend to pair themselves well with being a generalist, someone that can contribute all across the stack. So we try to give these folks that breadth of knowledge. So here's another quick video of just the curriculum that we built with some.

Previews in the first term you'll build projects on deep learning and computer vision. For example, you'll build a behavioral cloning project where you drive a car yourself in a simulator, kind of like in a video game. And then you use data from your own driving in the simulator to train a neural network to drive that car for you this is the type of project the cutting edge Silicon Valley startups are working on right now. And it puts you at the forefront of the deep learning and autonomous vehicle industry. You'll also build a project to detect and track vehicles in a video stream, just like real autonomous vehicles have to.

Do out on the highway.

In term two, you'll learn about sensor fusion, localization and control. This is hardcore robotics that every self driving car engineer needs to know in order to actuate and move the vehicle through space. In the localization module, you'll build a kidnap vehicle project which takes a vehicle that's lost and figures out where it.

Is in the world with the help.

Of sensor readings and a map. This is exactly what real self driving cars have to do every time they turn on in order to figure out where they are in the world. In the control module, you'll build a model predictive controller, which is a really advanced type of controller. That's actually how most self driving cars move through the world and use the steering wheel, throttle and brake to follow a set of waypoints or a trajectory to get from one point to the next. In term three, you'll learn about path planning. You'll have an elective month and you'll learn about system integration. Path planning is really the brains of.

A self driving car.

It's how the car figures out how to get from one point to another, as well as how to react when new obstacles enter the scene. I'm going to give you a sneak preview of how path planning works and this is something that nobody's ever seen before, so get ready. Path planning involves three parts. There's prediction, which is figuring out what the other vehicles are going to do around us. There's behavior which is figuring out what we want to do.

This goes on for a while, so we'll pause it there. The impact of this curriculum was bigger than we thought it would be. When we pitched as a small team this idea to Sebastian and to VICH at Udacity, there was a lot of skepticism that something like this was going to be successful. And the reason that there was that skepticism is that one of the kind of formulas that Udaci looked at to determine the impact of building a certain type of content was the number of open jobs available. If there was, for example, in web development, mobile development, all that good stuff, there was millions of jobs open. So it felt like there was a massive opportunity to impact the area. But if you were to in 2016 search for self driving car engineers or the different disciplines that exist within it, was kind of just Google. So it was very interesting just to see the instantaneous reaction that we had to launching this curriculum today. Over 14,000 successful students from all around the world. As you can see, probably the most exciting thing is to see what students have done with this sort of curriculum. For example, I learned recently that a set of our students here are building a self driving truck startup in India. Another set of students in South Korea are building a perception engine for self driving cars. Just a whole bunch of folks building truly amazing things. And not only that, they've gotten jobs at Cruise, Zoox, Waymo, Argo, all the big names and actively impacting those companies today. Now for the fun stuff. So we also decided to make that curriculum extra special. And we decided to do that by building an actual self driving car. And whenever I talked about this internally, udacity people ask me why? Why do we need to do this right? Isn't the curriculum just enough? Why go to the length of building an actual self driving car? And selfishly, some of it was just a personal want to build a self driving car. But the reasoning that I use is that what better way to prove to these students that are putting their faith in us that we know what we're doing than to build our own self driving car. And also what better way to collaborate with these students on an area that is really infantile than again by having this platform that students could actually run code on a car. So we decide to buy a car and we'll talk more about that in a second. But we set ourselves a milestone for our self driving car. It was to drive from Mountain View to San Francisco, 32 miles of driving with zero disengagements. It should be repeatable. It won't be zero disengagements every single time because otherwise we've got an actual self driving car. But in a short period of time, how much progress can we make towards this stated goal? Raise your hand if you've been on El Camino Real in that sort of region. Okay. So you probably understand it's got a lot of traffic lights. In fact, in our route, about 130 traffic lights. It's multi lane, three lanes, speed limit of about 40, 45, something like that. So it's fairly complex. But it's also got some constraining factors which is what we were looking for too. So it focused our tech efforts. This is the car we bought. You're probably very familiar if you follow self driving cars with the Lincoln MKZs of the world. They're everywhere and There's a reason for that in terms of the drive by wire nature of the vehicle and other stuff. And we outfitted a whole bunch of sensors, some cameras, some lidars, all that stuff. We also tried to build our own mount. We affectionately called this the Periscope. I don't know why it's in slow motion, but this was not our final design. Built all this from parts at Home Depot. Truly mvp. And then we got to work. The goal was to accomplish that milestone within six months. So we of course had to work fast. Assembled a dream team of folks that I'd worked with on different projects at Udacity that also wanted to come dabble in this. Folks that worked on the machine learning curriculum, robotics curriculum, et cetera. So this was one of our first days testing. And we did this at the Shoreline Amphitheater parking lot, which actually now is a very popular place to test self driving cars in the Bay Area because Google used to do it in the past. We saw a lot of weird stuff. For example, you'll see here, We saw what I believe to be a motorcycle gang. And we made progress. We kept iterating, kept building, and it started to come together. In fact, some stuff that we thought wouldn't work surprisingly just started to work. This is on El Camino Real. I'm in the back seat here. Stop lanes are based on grateful. So Mac discovered that we shouldn't have stopped at that traffic light. But we did. We resolved the mystery later. Let's go to the next video. And of course we learned a lot by going in this route. The different behaviors of drivers. One of the things that we were worried about is vehicles cutting us off. And when we say cutting us off, it means a vehicle pulling out in front of us, even a few hundred feet in front of us. You'll see here. We drove a little slow, 25 area. Turned out it was fine. And pretty soon it got quite boring. Car was doing very well driving itself. We built some cool algorithms to change lanes when necessary. Similar to you see with Tesla Autopilot. These days. We collaborate with some students on a traffic light classifier which was integrated into ROS there. And yeah, pretty boring stuff. So you can tell Eric was surprised that it was just fine. And we also had a penchant for building or for recording themed videos. Like you saw maybe from Elon Musk and the Tesla team with Paint It Black. We've got our own version of that. Eventually we became pretty confident, but we always wanted to test most of the day just to get the most learnings out of everything. This video was made at 2:30am driving from Mountain View to San Francisco, all 32 miles. Of course there's a backing track. Sam. Maybe want to turn it down so it's easier because there's less traffic. Right. This is kind of cheating and didn't count as the milestone. Just to be clear. You'll see that we eventually hit the 32 miles and Mac, who's in the driver's seat is pretty excited about that. It. Sam. And they hit it. But of course that didn't count because it's in the middle of the night and that's not going to be a very useful route. But it was awesome accomplishment just to even make it 32 miles with no disengagements when this traffic lights, lane changes, all that good stuff. But after four months, this is in the daytime. This began I believe at like 6, sorry, 7am we accomplished it. That small team had come together and built something pretty cool that could handle again, multi lane roadways, varying speed limits, traffic lights, objects, all that good stuff. And the thing that really brought this home to me is that the industry was now ready. It felt like this feeling I had had in software where someone in their bedroom can go and build something and launch it, almost feeling overnight could now not quite the same, but close to the same happen in self driving cars. But we'll talk more about what this led to in a little bit. Let's talk about open source challenges. We also got the same question, why do this? And it was clear to me that for something like self driving cars, which was so formative, we had to collaborate with students to figure out the best stuff. Because even the folks that were at Udacity were not necessarily the world's leading experts in these topics. So we wanted to use this hive mind of activity from around the world to teach the best stuff. So just through a period of a year, these are all the different challenges we launched. There was prizes and leaderboards and all this sort of fun stuff. The one that I'll focus most on today is using deep learning to predict steering angles. And the challenge was clear. It was that given a single camera frame, you have to predict the appropriate steering angle of the vehicle. If Anyone had read Nvidia's end to end papers in 2016, this stuff was all the rage. And it felt like one of those areas that was just begging for more exploration. And again, let's use this all these students from around the world to do it. And we did have students from all around the world. There was over 100 teams people self organized into these little groups to go and build this. And over the course of about four months we had a whole bunch of submissions all taking incredibly different approaches to the problem. We release data sets, validation sets, all that good stuff. Here you'll see the winning model. And I later found out that the author of this model actually went on to lead the self driving car team at Yandex, which if you've been following ces is doing some pretty cool stuff in self driving cars today. But you'll see this is on a route from the Bay area to Half Moon Bay, a very windy road. And you'll see that the prediction matches pretty closely to the actual, which is nice. And if you read his description of his solution, it's a pretty cool solution. And I think the most exciting thing was just the number of different approaches to the problem, all resulting in some awesome stuff. And, and again, in true Voyage fashion, we recorded a video of what this model performed like on our car. It wasn't perfect as any first model, and just the general approach of camera only driving had its faults. One of the main ones that we realized after trying all this stuff out is that of course a car, when steered by such an input, performs differently in a car than it does on your desk in a simulator or through prerecorded camera frames. So adjusting for those corrections that might need to be made is something that students after the fact added, which was pretty cool. So after all of these things, building that curriculum, building a self driving car, launching these challenges, it felt like it was time for something new. It was awesome to go and collaborate with all these students and it felt like I had to go build something. So gathered that same team that had built this curriculum and we said we're going to go build a self driving car. This is from my pitch at Coastal Ventures. You can kind of see the pitch deck there a little bit. Voyage is a new kind of taxi service. Our pictures change somewhat through time, but that's still pretty accurate. And we started what is now called Voyage and our goal really was that we wanted to again build a self driving car, but we wanted to do it differently. We didn't want to follow the same formula that we felt we'd seen from some of the other folks in the field. And the reason is that those folks have real advantages. Right. When you think about Google's project, of which I'm a big fan, they have this massive engineering pipeline of folks that want to go build a self driving car at today Waymo. But they also have a cash bank balance of billions of dollars that is hard to match. They also have the brand recognition of getting to work with Google and all that good stuff. So we just knew we had to think about this problem quite differently. And what motivated me is that today, as we all know, we have this incredibly broken transportation system. You step outside onto the roads today and I don't know about you guys, but I don't feel particularly safe when I jump into my car. We all know the stats. Over 1 million people have suffer fatalities on the roads today. Doesn't include folks that break necks, that injure, break bones, all that horrific stuff. It's also incredibly inefficient. We've again all observed this as we go about our day. Just the number of lanes that exist on a road today to account for peak traffic, the number of vehicles which have enough room for eight people, have usually one person in that front seat. I read a stat recently that only 7% of the average vehicle's energy usage is going towards moving the things that are actually in the car. The rest is waste. So an incredibly inefficient system. It's also expensive. The reason we see a lot of old cars on the road today is because that's at least today the most optimal and affordable way for lots of folks to get around and inaccessible. And you'll see why this matters to us in particular. Our goal is to introduce a new way to explore our communities. This is a video of one of our cars at a particularly cool place which we'll talk more about. And this is kind of our mission. And why now? Why is it possible to build a self driving car now? A number of factors that we learned during that Udacity experience, but some new as well. It feels from everything we see that sensors are now in this position which these sensors are now capable of. Level four self driving cars. The resolution, the range, the reliability, all those things that were necessary for an L4 self driving car are today ready. That didn't used to be the case. If you rewind to 2007 and look at the cars that were participating in the DARPA challenges, you'll see a lot of single channel lasers. You'll see the relic of the Velodyne HTL 64, the spinning bucket as it's called today, no one would have claimed those sensors already. But today you've got this enormous breadth of sensors that can take you that way. Compute is there. When we think about the recent rise in GPUs and whatnot, finally being able to have enough performance in the back of a car with the power constraints that you have, it's there. And talent. Again, this is not just Google. Today you've got all of these great minds from all around the world building this technology. So you're able to recruit those folks, put them to work on the problems they've solved in many cases beforehand. The reason I have yellow for computer vision, which is not a knock against computer vision, is because it's not quite there yet for a fully driverless self driving car. If you again rewound 3, 4, 5 years this would have been a red. But today with all the community and whatnot around computer vision, this is steadily getting to a green state. So pretty soon that'll be green. And of course then you'll have that perfect formula for level four driving. What we run after is ride sharing. We believe that the optimal way for people to move around is to be able to summon a car. But the thing that's suboptimal today is that you have to have a human driving you whenever you want to move around. Prevents the cost from being lower, prevents some safety issues, prevents some quality issues. We think solving that will mean these next generation way of moving around will come to fruition. But what we also see is that let's say we never remove the driver from the car, that a ride hailing network always had a human driver. You are inherently limited by the number of miles you can drive, which means that it will never replace personal car ownership. We'll never fix that. Fatality number. I talked about all of those things we must solve. So we think by having a self driving car that these next generation transportation networks will come to fruition. Our lead VC is a guy called Vinod Khosla, the founder of Khosla Ventures. An awesome guy who's done some truly world changing things. He has this quote which I'm a big fan of. Your market entry strategy is often different from your market disruption. Start where you find a gap in the market and push your way through. And this better communicated what I mentioned at the very beginning, which is that we should build a self driving car but do it in a different way. Because if we don't do that, we're going to fall into the same traps as many of the others that have died along the way. We have to find a way to do something different that we own and that we are really, really good at. And for us that was retirement communities. Hands up if you've ever visited a retirement community. Way less. There you go. Surprise. Lex Got to get you out to one. But these are just amazing places. And the reasons we choose retirement communities first to deploy our self driving technology in is for these four reasons. They are slower. The speed limits in these communities tend to be far slower than you'd see on public road. Much calmer roadway. When you visit these locations, I liken it to listening to a podcast at 0.75x. Just very constrained, very slow and a little boring from time to time. But you've also got these heartfelt transportation challenges. We hear from these residents all the time about how transportation is a pain point and that their only option is a personally owned vehicle. These folks know in many cases they shouldn't be driving, but because they don't have an alternative, they still drive. We hear from folks that put off much needed surgeries, hip replacements, things like that because they don't have a friend in town who's going to be able to move them around. We hear from folks with vision degeneration that they just don't see a way that they'll be able to move around and keep that quality of life that they've been able to have. Folks gripping steering wheels for extended period of time. All these challenges that felt like the best first place for a self driving car to begin and a clear path to customers. We see that on the roads today. Ride sharing on public cities and whatnot is a particularly brutal battle. A race to the bottom in terms of cost. If we every retirement community in the country, meaning the transportation networks there, that would in and itself be a very valuable, very valuable business. One of my favorite passengers is Anahid. She came to visit us recently and gave this quick speech about why self driving cars matter to her and her community. Not only that, but we're concerned about safety. I was on the road and with one of the drivers, a car is turning the wrong way. Let's talk about our first community. This is the villages. Whenever I show this slide, people are astounded by the number of residents in a community like this. Over 125,000 and growing over 750 miles of road. And what we have in this location is an exclusive license to operate an autonomous vehicle service. This is one of our other beliefs which is that by partnering very deeply with the community it means that we're able to deliver a better service and that we're able to grow a more reliable business. We won't have entrants and competitors from all of the other self driving car companies in our communities. What we actually do in exchange for that Exclusive license is grant these communities equity. Because if we win, it's probably in fact highly likely as a result of those communities. And the addressable market of transportation in these regions is massive. These residents tend to be, as a lot of seniors tend to be quite affluent, which means that they have some disposable income when it comes to being able to pay for ride sharing services and other things like that. So we find that, that, that recipe is absolutely perfect here. And we're launching, and have launched passenger services to these residents. Gotten a lot of awesome feedback, learned a lot about the needs of providing ride sharing for senior citizens. Just some quick stats. This is from my series X fundraising deck. Just about the size of the senior market. Again, this is the first place we go, but, but you can get a feel for just how large this transportation market is. Today there are 47 million seniors. That's growing by 2060 to over 100 million seniors in the U.S. the total addressable market for just seniors is incredibly large. 2,500 plus communities. All that good stuff. And this is how we see the world, the landscape of potential deployments. You've kind of got a lot of the big guys focusing on that bottom left quadrant. They're focusing on large cities. And it makes sense because it's playing to their unique strengths. It's playing to their ability to deploy thousands of cars, tens of thousands of cars. It plays to the strength that they have at least some patience or ability to have more extended timelines when it comes to building this technology. But for a startup like us that fights for survival every single day, it means that we have to do things differently. So we focus on that top right, right quadrant there, what we've kind of coined as self contained communities. These places are simpler, slower, but they also have this ability for us to have that exclusivity that I talked about. And there's some others, of course, that we play in. Whether it's the senior market or maybe even small cities and things like that. Let's talk about autonomous technology. So just to reiterate, why do we deploy in retirement communities? Slower speed, simpler roadway. There is a central authority. These places tend to be run by private companies, which makes for a quite unique relationship in a very positive way. It means we can deploy faster. It means we have the potential to have more impact in these regions. It also turns out that retirement communities tend to be located where there's ideal weather for self driving cars. Think about Arizona, Florida, et cetera. We have a world class team building this at Voyage from All the major of programs out there. And that makes our lives infinitely easier. One thing that also makes our lives easier is the sensor configuration of our car. We've intentionally made this decision that we're not going to focus on optimizing for cost today, but to optimize for performance. We want to get to truly driverless sooner than most. And one of the easiest ways you can again make your life easier is by optimizing for high resolution sensors. At the very top of the vehicle we have the VLS 128 which is a 128 channel LiDAR that's capable of seeing in 300 meters in 360 degrees, many other different lidars on the vehicle to cover different certain blind spots. Altogether we process 12.6 million points per second. And that just looks incredibly high resolution. You'll see our car at the bottom there and that's the raw point cloud output that we see in the world. We run towards level four. And for us what that means is that if you're building a demo self driving car, kind of like we did at the Udacity project, you may focus on just the top four items that top row you may focus on perception, prediction, planning and controls. And it turns out you can build very impressive demo quite quickly by just focusing on those things. But of course those things fall apart whenever edge cases are introduced, which happen all the time. So we've spent a ton of time on all the items here because again our goal is to build not a demo, but a truly driverless vehicle. We also have a emphasis on partnerships because what we've noticed in the self driving ecosystem is that there's not just more self driving car companies building the full stack. There's now folks getting into simulation, to mapping, to middlewares, to tele operations, to routing, to sensors, of course, and a ton more. So we make our lives again easier by partnering with companies like this so that we don't have to spin up a simulation team or we don't have to spin up an operations team to go map the world. We can just work with these very cool companies. Let's talk about one unsolved problem which fascinates me. It's to do with perception. And you probably won't be able to notice this unsolved problem from just this picture, but maybe if I add some annotations you might. Foliage, trees, bushes, whatever you want to call them. You may have seen some quotes in the media about some popular AV programs struggling with such foliage. For example, cruise cars sometimes slow down or Stop if they see a bush on the side of a street or a lane dividing pole that was in the information wrong way. This one. Uber's self driving car software has routinely been fooled by the shadows of tree branches, which it would sometimes mistake for real objects. Insiders say that's Business Insider and even Voyage. There's only one hard stop on the way. The culprit is a bush two feet high that protrudes into a lane from a street median, which Voyage considers a possible threat. Voyage may trim it and we did, but we don't think that's scalable. Or maybe it is, I don't know. But we at the beginning of 2018 decided to solve this problem. So of course all of this resides in the world of perception area of particular fascination for me. We're sharing these slides, but these are just some of the papers and research that we see going on that intends to solve those sorts of issues. One of the reasons you've seen those programs, including ours, be particularly sensitive to foliage is because from a perception perspective, one of the most well known way to detect objects is to utilize the map. If you have this map and you effectively it's simplifying to a certain extent, but subtract objects that aren't in the map and then use that as a way to understand what's in and around you that's dynamic, then of course you'll end up with decent representations of cars and pedestrians and whatnot. But if foliage grows, which it does, trees, then that's going to extend out from the map and mean that that particular bush is now an object in your path. These networks here, of which these are all neural networks, don't use that same technique. They don't use the map as a prior. Instead what they do is take of course, this 3D scan of the world and then take a more lear approach to the problem. You'll have tens of thousands, hundreds of thousands of labels of cars, humans, et cetera, and then these next networks will be able to pick these ones out. We're particularly fascinated by Pixel, which came from some great researchers at Uber. Atg. Voxelnet came from Apple spg. I've heard our engineers talking a lot about Fast and Furious recently, which merges together perception, prediction and tracking into a single network, which is pretty cool. And point pillars, which I think came from the newtonomy team recently. I think Carl is speaking soon. Right. So just in general we see a whole bunch of work going out there to solve these issues. The other one that these sorts of networks solve, which I also find particularly fascinating is that if you use traditional clustering algorithms, what you might see is that if two people are stood next to each other, traditional algorithm will cluster as this one object which when you're trying to move away from those edge cases and build a truly self driving car that's a non starter because pedestrians are the most important thing you can probably detect. And detecting two things as one thing is not going to cut it. And of course it does that because it's a dumb algorithm, it's not trained on any sort of information. But these networks again are very very good at understanding the features and perspectives of humans, even if they are in crowds and whatnot. And that then helps all your stack downstream because if you have accurate perception information about objects in and around you, your predictions are much better, your tracking is much better and ultimately how you navigate the world is much safer. I'm also particularly fascinated by reinforcement learning, which I know Lex is as well. If you've read Waymo's recent work on imitation learning, I think that's particularly cool. Another company we track quite closely just because they do amazing stuff is Wave. Trying to build an entirely self driving car powered by reinforcement learning. Think about disengagements as rewards and things like that to be able to tune that to better performance. Also just areas of learned behavior planning ultimately fusing rules of the road with more learned behaviors. The ecosystem. I think it's this area that is thriving today. Seeing just how many folks are diving into not just the full staff, but building tools and building other really important parts of the stack. The maturation of sensors, not just higher resolution lidar, but things like 3D radar. We get pitched all the time from these companies and it's clear to see there's been a rise in volume from all these great efforts. Lessons learned. Now that I've been building Voyage for two years and and prior to that four years at udac. What things have I personally learned that are not technical in nature? So many things. So these all may look like cliches but I promise you they all came from lessons which were really really painful in the moment. Don't be intimidated. So the thing that I feel happens a lot in self driving cars is that because it started in this very academic sense, meaning Stanford, Carnegie Mellon and whatnot that but it felt like to break into the industry you had to also go through that same path. You had to get a PhD in something and really go the path that was well trodden. But I think that only takes the industry so far. I think it's really important that we get folks from all different backgrounds, all different industries to come contribute to this field. Because if we don't, there is no driverless. It can't happen in that isolated bubble. It needs to be extended out. So. So don't be intimidated by those things. Understand your limitations. This is perhaps more of a kind of CEO lesson for myself, but I think when you're building out a company from one person or five people to today were 44 folks, you cannot do everything. And it's really important that you build a team around you that is able to do what you used to do, but do it 10 times better. I probably didn't spend enough time building out that team until we had some challenges our way when it comes to that stuff. Be proactive versus reactive. I think it's really crucial, again, when you're building a company to try and predict what's going to happen next. Because if you're reactive, you're constantly two steps behind what other folks are doing. Get out of the way. I think a lot of folks, again, perhaps overstay their welcome in certain areas of the company when they should just say, okay, I've got experts now. I can just, just step aside and let those folks do what they do best. And speaking of which, hire the best. It's really easy when all this pressure's on. When you're building a company to kind of sacrifice when it comes to your culture, when it comes to hiring, it's really crucial that you find folks that are not just the best in their field, but are the best match for your company. And always be curious. I think it's always one of the things we believe in at Voyage is that it's important that knowledge is not isolated to just one person. That that knowledge should be spread throughout the company. Because even though it may feel like oversharing or over communicating, what that knowledge may mean for someone that has a particularly unique background is they may do something incredibly cool with it. They may build something that totally transforms our company. So that's about it. Jump to questions if that's helpful. Cool. That was great. Please give a big hand. How did you identify retired communities as the target market? Prioritize. Yes. So retirement communities for us was actually, there's a really long story, but I'll trim it down a little bit. So when we were starting Voyage, Sebastian Thrum was very helpful in helping us start this company. And of course, as kind of naive founders of a company, we were like, let's just take this el Camino thing and put it on everywhere else that looks like El Camino and just do that over and over again. But he cautioned against that, and very wisely so, because again, you're nothing special compared to the other self driving car companies out there by doing so. And in 2009 he had really advocated to Google leadership, et cetera. Larry Page, that retirement communities for self driving cars might just be the best way for Google to go about deploying their self driving cars. And I can understand why. I think the Google folks were Google. Right. We're not just about retirement communities, we're about the world, like level five or nothing. Right. So he got some pushback, but he did some research in that process, met some folks. So when we were starting, he's like, you got to check out these retirement commuters. So we did, we went to visit and eventually we got there. So we wouldn't have got to that point without, without Sebastian pushing for that. Just to follow up on the question of retirement communities, the question is, do you ever think about the other collateral issues? Especially the retirement community would have to get into a car. Yep. And how exactly would they interface? Like somebody wants to make a call to have a car come to their, wherever they are and they have to move from A to point A to point B. So how did you ever think about all these issues that are very germane? It's not just a vehicle moving on its own, but these are all collateral issues. How do you plan to address this? That's a good question. So the way we think about this is that today we've intentionally focused it on a segment of the market which is called the active adult communities. These folks tend to be able to, you know, go into their own cars or into a taxi, open the door, sit down without the need for any, you know, assistance when it comes to that. But they may have vision issues, they may have other issues that prevent them from driving. Perhaps, for example, we hear a lot that folks feel really uncomfortable driving in the evenings. They feel comfortable driving in the daytime because their vision supports it. But when it comes to evening time, they, they have this mad rush to get home. But there is that other market which you're talking about, right. Which is folks that just need that helping hand towards getting to the car. And one of our beliefs as a company is that the senior market, like I had in that slide, is surprisingly large. And what that means to us is that we think we can own it. We think we can be that company that any senior citizen in that situation thinks, oh, I should call Voyage because I need to get from point A to point B instead of thinking I should call Waymo or Cruise or any of the folks that are going to go after the general big market. They'll think about Voyage. And the reason they'll think about it is because we'll deliver a product to them that is meant for those folks that is designed for their use cases. It may be that actually if they're going on a long trip, let's say they're traveling 50 miles the first mile of that trip and the last mile of that trip may involve a human helping them into the car and then dropping that human off somewhere else to go do that all over again. It may involve crazy robots that help people from their cars. We've heard from folks at Toyota that are building these bag carrying robots and other things that may assist seniors from getting to the cars and whatnot. So I think that's why that market for us is particularly exciting, because it feels like you can deliver these tailored products that would enable us to be the market leader. But today we focus on active adult. But who knows where you go next. Can you talk a little bit about how you determined your final sensor suite? Yeah. So the truth is, it's never final. So we think about generations of vehicles. So we have our first generation vehicle, which was a Ford Fusion, had a single Velodyne HDL 64 in it, bunch of cameras, radar, and we set some milestones based on that vehicle and we accomplish those milestones. And then once we reach kind of that, the max in which we're able to take that vehicle, we then say, oh, we need to bring on our G2 vehicle, our second generation vehicle. So we did that and we said, okay, we have these certain goals in mind, which are pretty lofty and pretty ambitious. We need incredible range, incredible resolution for these things. And actually what we've discovered is that in our particular communities, going at the speeds that we're going at radar isn't particularly useful. So we don't have radar in our second generation vehicle, for example. But I'm sure that when we go to that third generation vehicle, there'll be other driving factors, is that we work backwards from the milestone to say, what do we need on this vehicle? It may be cost in the third generation vehicle. Right. We may say that, hey, we need a more affordable sensor suite than what exists in our second generation vehicle. But they're driven by technical requirements and that means that we are able to really marry the two with the vehicle. I was curious when you showed the student LED or when you showed one of the students in your front first practice car had developed a traffic light sensor and then you showed later on that you were getting student input for deep learning models for steering wheel turns. I was wondering what your system architecture looks like in terms of the kinds of perception that you take in, how modular it is and to what extent deep learning algorithms have played a part in those different parts of the. That's a good question. So I really encourage folks to get familiar with ros. So ROS has always been this kind of playground for roboticists of all different types of robots to be able to try things out on robots. And ROS1 is particularly notorious for kind of hacky and hobbyist types of projects, but it's not meant for production. ROS2 though, which is in kind of an alpha release state, state is definitely meant for more production oriented things. And the reason I mention ROS is because it has this awesome architecture which lets you plug and play what they call nodes and be able to experiment with different approaches to the problem. So for example, what was running that deep learning model predicting steering angles effectively replaced our more rules based planner and perception engineering. And we just plug the output of that to of the steering angle straight to our controller to just actuate the vehicle. And ROS is particularly good at those sorts of architectures and it's all open source, so you can do some cool stuff with it. Can you tell how you handle the liability insurance related for passengers for your vehicles? Also how we handle insurance question. So we have a pretty cool deal with a company called Intact Insurance. And the idea is that insurance in the autonomous age is going to be very different than insurance today for human drivers because there's different risk assessments and whatnot. And one of the ways that we're able to prove to these insurers that we're good at what we do is actually sending them data. We send them data from our cars as we drive, showing that as we move through the world, we accurately detected things and planned around things and all that good stuff. And then they use that data to inform our rates of insurance. I think that the future actually of insurance will be on a similar lines, but perhaps more extreme, where for example, the rates will change depending on the complexity of the environment. If we're just driving down a straight road, completely straight, and there's zero vehicles around us, our insurance rate should be super low, right? But if we enter a city center and there's thousands of people and cars and all that crazy stuff, stuff Our insurance rate should just rise almost instantaneously. So we're partnered with someone today that insures the passenger, the car, sensors, all that stuff. But I think there's a lot of room for innovation there too. Did you have any problems like onboarding the retired people initially where they're like, you know, skeptical, scared? And then the other question is, what are the major missing pieces in terms of computer vision to achieve L4? What was that last missing pieces between computer vision in computer vision to achieve level 4 self driving. Gotcha. So one of the more interesting insights I think we had about retirees is that again, in my kind of naive State back in 2016, my general feeling was retirement communities might not be the first to adopt this technology, right? Because they may be slower to adopt new technology, might be scared of the technology, all those sorts of things. And to kind of validate that, I went to talk to some senior citizens because I talked to my own grandma, she hates self driving cars. So I was like, that's not a good sign. But went to talk to these folks in these sorts of locations. And the really interesting thing we learned is that traditional consumer software or devices, yes, there is definitely a lag in adoption with senior citizens. And that's proven in many studies, many stats that senior citizens are slower to adopt the Facebooks of the world or the, the Instagrams or the WhatsApps, all those sorts of things. Cryptocurrency, I don't know. But that's because they have these very well defined processes that they've had for most of their lives, right? Instead of using Facebook, they call someone up and they have a chat, you know, a conversation with someone about their day or stuff that's going on, or they don't share a picture on Instagram, they physically mail a picture or something like that. So to change that behavior is tough, right? Because that's a behavior that is fundamentally different than what they're used to. They have to log onto a computer, go to this weird Facebook thing and share pictures with thousands of people. That's weird. But the difference between that and a self driving car is that our experience is no different than the car they're used to. It just turns out it's being driven differently, right? Like they see a car, it's the same similar form factor to what they're used to. They open the door, they sit in the backseat, okay, there is a button I have to press to say go. But it's pretty similar to what I'm used to in my past. I don't have to learn a new behavior. I don't have to change something that I'm used to. So that was our first learning. And then also they actually really don't care too much that it's autonomous. They have very. When I'm in the car, I'm quite curious and enthusiastic about the technology and want to tell them about, I don't know, lidar and deep learning and perception. And they just, you know, don't want to hear any of that stuff. And it kind of dawned on me that the reason that is is because what they senior citizens have witnessed over their lifetimes is far more dramatic than I have. Right. Like we've. Our oldest passenger was 93 and she told me a story about how when she was very young, she remembers literally moving on an almost daily basis in a horse and car. So when you talk about like self driving cars to those folks, like, they just, you know, they couldn't care less because between that period and today they've seen the birth of like flight planes everywhere. They've seen car proliferation, they've seen scooters now they've seen like all of this crazy subway systems. So a self driving car to them is like, oh, that's cool. I just wanted to move me. So that's our biggest learning there. The question was computer vision. What needs to happen between now and level four? Yeah. So I think the holy grail. So if you had perfect perception, self driving cars are solved. If we knew every object that was on the road in and around us within a reasonable distance, self driving cars are solved. False positives are accepted today, which I think is good. But you really want to minimize false negatives, right? You want 0 false negatives in the world. And I think that's why we still have a tiny bit of work to do. Because when you think about the reason for a test driver being in the vehicle, well, perception feeds everything downstream. So if you miss an object, misidentify an object, any of that sort of stuff, then that effect causes the whole stack downstream to become quite chaotic. That's why I'm excited about all those networks that I talked about. One of the other things we believe that helps us minimize false negatives to non existent kind of status for us is that we band together multiple networks. So we don't just rely on a single layer of perception. We say different networks have different strengths. For example, Voxelnet is particularly good at pedestrians, but Pixar is not so great at pedestrians because it's from a bird's eye view where pedestrians are quite thin and whatnot. So let's ban those two networks together and let's also band together some more traditional computer vision algorithms that may not be processed on the entire 360 scan, but maybe process on a small sample, maybe at the front of the vehicle, for example. So there's just lots of little bits and pieces like that to go through to minimize the worst case scenario, which is a false negative. But it's clear when you see Waymo and whatnot that they feel very, very, very close to that sort of state. You mentioned that weather was one of the main reasons this was a great place to start. Can you talk about hurricanes? Yes, it was funny. I got a question recently from Alex Roy. Me and Lex are just talking about, okay, in the event of a hurricane, right, let's not talk about the technology second. But in the event of a hurricane, like we've all seen those pictures of people getting on the freeways and trying to get out of the path of the hurricane, right? How is that going to work in a world where self driving cars are everywhere and personally driven vehicles are maybe more of the smaller. So I don't quite have an answer to that yet, but I think it's an interesting kind of thought problem from a technology perspective. The really important part of weather for us is remote operation. So inside every one of our. Sorry, all of our vehicles have a cellular connection, right? And each of those vehicles is connected to a remote operator that's sat in somewhat close proximity to that vehicle. And that remote operator has a few jobs. One is to just ensure the safe operation of the vehicle, make sure that vehicle is doing as it's intended to do, all those good things. But another is to make sure that the operational domain that we are currently operating in is the one that it's designed for. So all these different camera feeds are being live streamed to this remote operator. And if there is sudden downpour of rain, that remote operator has the ability to bring that vehicle to a safe stop until that rain shower disappears or whatever, or hurricane, whatever it may be. But there are companies. I was pitched recently by a company that's building weather forecasting on a scale that is not really used today, but really microclimate. So thinking about just this small subsection of the villages, predicting and understanding exact weather within those regions and then having web hooks to tell you or us voyage that that's about to happen. So there's a lot of cool stuff happening there, but remote operators currently kind of the eyes and ears of our cars to prevent that sort of issue. So please give Oliver a big hand. Thank you very much.

Thank you, guys.

Sam.