Aurora's Chris Urmson + Nuro's Jiajun Zhu | The Road Ahead for Autonomous Driving

Hi everyone. Welcome to Grey Matter, the podcast from Greylock where we share stories from company builders and business leaders. I'm Heather Mack, head of editorial at Greylock. Today, Greylock general partner Reid Hoffman talks with two entrepreneurs who are driving innovation in autonomous vehicle technology. His guests are Chris Urmson, who is the CEO and co founder of Aurora, which has developed self driving technology for trucks and passenger cars, and Jay Z who is the CEO and co founder of Neuro, which has developed driverless delivery vehicles. Autonomous driving is among the most exciting applications for artificial intelligence. It's also among the most challenging to actually pull off from the capital intensive combination of hardware and software to the complex coordination between multiple stakeholders to ensure safety. In this conversation, Reed, Chris and Jay Z take stock of where the field is today and discuss how major advancements in the past few years have made the reality of wide scale autonomous driving closer than ever before. This interview took place during Greylock's Intelligent Future event, a day long summit featuring leading experts and entrepreneurs in artificial intelligence. You can watch the video of this interview on our YouTube channel and you can read the transcript on our website.

Greylock.Com and so now I'm delighted to invite two other friends up on stage. You guys should start coming. CEO and co founder of Aurora, Chris Remsen and CEO and co founder of Neuro jz. Two of the folks who I also learn from. Thank you. Part of our selection of the folks that we would have come join us are people that we learn from intensely. So let's start with the kind of the current state of, of what's happening on roads and autonomy. I guess I'll start with you, Chris. I think most of the people in the room will understand that actually we're already on the road, but they may not realize how much of it is actually already pragmatic. And I kind of got that because of the like sitting in the truck, going on that path from Dallas towards Houston, looking at the server closet, looking at all the rest going oh my God, like you can see it but say a little bit about what the current like today is happening.

Yeah, I think one of the things that's happened in the self driving space is over the last few years we went from a mode of hey, it's all solved, it's done tomorrow, we're going to ship it next week to boy is this ever really going to happen. And I think both extremes were and are wrong, right? That today there are vehicles on the road, you can get a ride in a cruise vehicle, you can get goods delivered By Nuro. You can get a ride in a Waymo vehicle with nobody behind the wheel. You can come down to Texas. You can see one of our trucks driving between Dallas and Houston. And so this is happening today. It's happening in a meaningful way. And we're pulling loads. Like if you got a FedEx partial that got shipped from Houston, there's a good chance that it sat in one of our trucks on the way there to you. So I think it's incredibly exciting. It's going to take a little while as we do things in the physical world and we put in place the supply of vehicles and integrate with them and do all the work that we have to to make sure these things are safe. But it's coming along really well.

We'll come back to safety in a bit. But also jz, same question in part. We have one of the vehicles here that actually has California tags on it that kind of rolled in, but same thing. What does today look like looking out for the next couple months?

Yeah, definitely. I completely agree with Chris. I still remember, probably 13 years ago we started working together on Celvi. It's been a long journey. I think this is definitely already in a zero to one we've passed the one. Now it's about scaling. So as Chris said, we already have self driving cars on the roads. Different companies have different services. Last year we have the first driverless pizza delivery in Houston. And it was just amazing to see customers ranging from 3 year old all the way to 80 years old using a robot getting their pizza from Domino's. So this is definitely happening. I think now the industry is trying to figure out how to grow, how to put more vehicles on the road safely and grow this as a business.

You know, part of what makes us at Greylock, as a just amazingly delighted investor in both companies and was part of our discussions early on is that you were both very sophisticated about how you approached safety because obviously safety is one of the important things. I'll just invert it just so we're not always doing the same pattern. I'll start with Jay Z and then go to Chris, talk a little bit about how safety is fundamental to your architecture from ground up and what you're doing.

Yeah, maybe I'll talk a bit more about our unique applications. So because we focus on goods transportation, there's no passenger in the vehicle. And by not having a passenger vehicle, we get the opportunity to really design the vehicle from the ground up to optimize for safety for other road users. So not only we try to design a vehicle that is very neighborly friendly, but also it's a bit narrower so you have more space around you. You can avoid accidents more easily. And because there's no passenger, you can do things on the vehicle that you cannot do on a passenger vehicle. So for example, at the front end of the vehicle we have a external facing airbag. So this is in case there's an accident, there's a collision, the vehicle will detect that and Deploy the airbag 30, 40 milliseconds right before the collision. So this is something that we think is going to be really, really good for safety. Another example also because there's no passenger in the vehicle, we can decelerate the vehicle much more aggressively that you cannot do if you have a passenger inside. So these are examples of things that we had basically at the first day when we start designing the vehicle, try to leverage the fact that we don't have passengers inside.

And for us, we drive passengers and we drive giant trucks. So we're neither skinnier nor can we stop as aggressively as the airbags in the front. Well, I guess we could, but I don't know that it helps with a 70 ton truck or whatever we're driving down the road. And so for us, we have to think even more holistically about safety. We embed it right in our culture. If you look at our company's mission is to deliver the benefits of self driving technology safely, quickly and broadly because it was so important to put it in place up front. And then we're one of the few companies, I think maybe the only company that has actually shared how we're going to convince ourselves the thing is safe to be on the road. And this big structured argument, we call it a safety case framework that breaks down the technical reasons when things are working, why they'll be safe, the technical reasons why when things break we'd still be safe. And then all the organizational and process stuff that we do around that to help make sure that the system's cost constantly improving and that we have controls in place. And so we really just don't think you should be doing this without doing that hard work. And as we look at this technology, it's going to shape transportation for the next century. And so investing in that foundational part of this so that we don't kind of foreclose the future, I think is critically important.

Now related to safety is regulatory and regulatory has good and regulatory has bad. What are the places that each of you would kind of suggest nudge, say here is the kind of regulatory environments that governments, especially us, especially California, we've all interfaced with that would say do more of X, do less of Y. Right. Because both of you are, we're totally in the safety, like regulation is good for that, et cetera. Like you're pro regulation in that. But it's how to do it intelligently is the thing. So Jay Z, you want to start?

We engage with the government very, very early. Even before we came out of stealth, we were working with Department of Transportation. And I think there has been a lot of success on that engagement. So we became the first company. This vehicle was the first vehicle that got exemption from.it doesn't have steering wheel, brake pedal, but it's road legal. We also got the first commercial deployment from California, which is great. And we were also, I think, the first driverless pizza delivery in Houston. So to us, our mission is to better everyday life through robotics. So it's really, really important for us to really work closely with the policymaker to help them see the benefits. And I think so far has been well received. There are areas where we think we can potentially move even faster. So I think maybe not on this vehicle, but if you look at other vehicles on the road, we still have tiny little side looking mirror. And this is one example where we have to work still according to the current regulation and to make these vehicles really following the current code. But this is not necessary for autonomous vehicles. And I think these are areas where policymakers could really help us.

I think when it comes to regulation, it's important to understand the landscape as it is today. In the vast majority of the US if you have a vehicle that you're confident in the safety of, you could put it on the road and operate it without having to do anything else. So that is a real competitive advantage for the country. And so when we talk about regulatory change, it's really around the edges and it's to help maintain that advantage over time. And so much like Neuro, we've been engaging both at the federal and state level since day one. And it's about helping the regulators, helping the lawmakers understand the promise of the technology, understand really what the challenges are with the technology so that they can be informed. Because the worst thing that happens is something bad happens and it will, right? We're talking about the real world and some congressperson calls the regulator and they have to make a knee jerk response in that moment versus they're informed, they understand the situation, they understand the issues and they can respond in a thoughtful way. And so that's been our strategy over time as I think about what could be better. We would love at some point when the societal benefit of this technology shown that you get protections for litigation that look like the airline industry. That would be great for business, great for society. We would love to see more nationalized, harmonized regulations. So the way the regulatory stack works is the federal government regulates the safety of the vehicle, the states regulate the safe operation of the vehicle. Normally those are relatively deconvolved. Obviously when you build a self driving vehicle, they become quite convolved. And so it'd be great that the rules of the road, the safety expectations were consistent across the country.

A lot of folks in the room will already know because they'll have some ties to the AI world and so forth. Like what the importance of simulation is and how do you wrap that in your process of learning and all that. So we don't go into that. Let's actually go a little bit to the tactile hardware parts of it. And one of the things I knowing both of you very well and us how we can all, all having collaborated on this is there's a little bit of a debate of well, you should just do it with cameras. No, no, no, actually lidar and all the rest of these things. And actually, in fact my own point of view, which I think is, I'll state it for myself, but I think it's reflected in the things you guys also believe is like, well, why not have maximum safety? If I was driving a car and I had lidar too, that'd be great. Because let's make it safer.

Sure.

I have perfectly functional eyes and responsible. Let's make it safer. Say a little bit about the kind of sensors, sensor fusion, the approach for making this not just as good as what we have, but so much better than what we have. Jay Z, I'll start with you and then Chris, we'll go to you.

Yeah, sure. I completely agree with you, Reid. So we have laser, we have radar, we have a camera with, we have thermal camera. Some of the sensors are built in house and we really believe that this is the right path for level four autonomy. And I think one, maybe one insight was that I think people tend to look at, oh companies like Tesla, what is their approach? And, and I think there is a bit nuance here that I think the optimization function, if you think about building a product for consumer vehicle versus building a fully autonomous fleet, maybe you have a, you know, operate the fleet. That's your, that's your business. The optimization or the objective Function could be very different. I think the cost of the sensors could be different. You know, how, how fast you're going to use all of these sensors, how fast you can replace them. And I think all of these things become very, very different. So I think for our business where we operate a fleet of autonomous vehicles for goods delivery, in this case, we really believe that using the most advanced redundant sensors to maximize for safety is the right pass.

Yeah. So we're going to break news today and we're going to announce that we're ripping everything that isn't cameras off of the Aurora vehicles. No, we totally don't believe that. Right. We agree completely with the way ChatGPT thinks about this, that I think, you know, I can't remember who said it from the Tesla camp, they said LIDAR's a crutch. And you know, and in my mantle mind or my model, engineering is all about cheating. And so anytime you can cheat by using better capabilities, better technology, you should do that. It allows you to solve the problem more quickly. It allows you to solve the problem in better ways. And again, as we think about whether it's a multi thousand pound light vehicle or a tens of thousand pound truck driving down the road, having redundancy in the way you understand the world, having the robustness that comes from seeing at different frequencies and, and thus different penetration of weather, different ways things fail, it's really the only way that you're going to get something robust enough that we could really trust it. I am one of those people who bashes human driving. 40,000Americans killed last year and that number's going up. But the flip side of it is it's something like one every 85 miles of dragon driving, which is a really impressive statistic. And so if we want to hit that level of performance, that many nines of reliability and robustness, we have to pull kind of all the tricks out of the bag to be able to do it.

Well, that kind of actually led to the next question that I had here, which is what should be the expectation of kind of the benchmarks of the okrs of what is safe enough, what is efficient enough for kind of full pedal of the metal. We should be deploying this stuff as fast as possible. We already know that we have enough safety, we're on the road, we've got various things happening, Texas, other places. But what should be some of the concepts where people say okay, that's the measure because there shouldn't be zero accidents, that's foolish, irrational and self destructive. So what should be the way that people think about. Chris, we'll start with you.

Yeah. So first it's got to be commercially relevant. Right. There's not a whole lot of point scaling something if it can't do something useful in the world. And I think we're both fortunate that we're working on applications where there is immense value. Right. We have tremendous shortage of drivers. We've just seen the impact that partly led to with the supply chain challenges. So there's got to be both commercial and social value there. And then on the safety side, you're right, we can't let the perfect be the enemy. The good. Right. And over time this technology is going to improve. As we get more experience the road, it's going to improve. We're going to learn more about how things fail. And so we need to find an acceptable level of risk that is, you know, is not unreasonable. And by the way, that's the bar that Department of Transportation uses that you see you're not creating unreasonable risk on the road. So for us, we think about how do we know the Aurora driver is going to do the correct thing? And I would pause it for you. Imagine you're driving down the road and a vehicle swerves into you. This is actually one of the very common ways that trucks get in accidents is actually another vehicle swerving into them. Well, it's probably not okay for you to be oblivious to it and just have them hit you. Right. You should probably take some kind of responsive action. But it's also probably not okay for you to have like drive off of the road to avoid it. Right. There's a lot more risk that comes with that. So there's some correct behavior which is move away up to a limit and slow down. So as we think about it, it's kind of spelling out some of those underlying simple, understandable, guiding principles and achieving a level of performance that is comparable to a good human driver. Yeah.

The only thing that I would add is I think one thing that I learned from building Neuro and also previously working with Chris at Google was I think building an autonomous system that is in a real time safety critical environment in the physical world is really, really hard. And in order to do that, it's not just, you know, the AI part needs to be really, really good, but you also have to develop this really amazing foundation in the company from requirements all the way to validation to go through the entire process that allows you to have the confidence in the safety performance of the product that you're building. And to me, I think that is a pretty unique aspect of autonomous vehicle technology. And I think Chris said, which I agree not to introduce unreasonable risks when we deploy. How do you actually measure that? How do you actually quantify that? I think this is really the challenge that we're all working, trying to solve.

What are some of the most challenging parts of the environments that you, each of you is navigating? That is part of, like, this is why this is here and now, but also the result of so much work and the work that you're still doing. And part of that just to give folks an idea, because one of the things I thought was funny when we did the investments is watching how easy it was to get someone going to 70%. And then the rest of it starts, started getting much, much harder. And both of your guys focus on that. Both of you taught me things around, okay, how are you dealing with construction cones? How are you dealing with tunnels? How are you dealing with. And those are the questions I started asking people.

So with us, as we think about trucks driving at freeway speed, the amount of kinetic energy involved is large. And so to be able to kind of slow down and avoid things, you have to look out, you know, getting on for half a kilometer, 400, 500 meters. And it turns out that's hard. Right. You either have very narrow field of view cameras, or preferably you have narrow field of view cameras plus amazing sensors. And in our case, we're developing this special technology, first light, which is a kind of lidar that can see further than the pulse lidars that almost everybody else uses. And it allows us to both see further, it allows us to instantly measure how fast things are moving so that we could figure out how much of a threat they are. And that allows us to react meaningfully sooner. In the case of what we've been looking at, our sensor versus some off the shelf lidars, we see about two football fields further than they can, and that is about seven seconds sooner that we get to react, which is just magic.

Yeah. I think the long tail edge cases are also evolving over time. So I think maybe 10 years ago, it's very much kind of focused on perception problems. Right. I remember we probably spent a lot of time dealing with like leaf blower or fog, you know, or just reflective signs. I think nowadays, I think the edge cases for us, we focus a lot on low speed neighborhoods, maybe the opposite end from the high speed road, where you have to see very far, we don't have to see very, very far, but you have to be able to handle a lot of Unpredicted agents in the world. So for example, kids riding a bike that could be very, very unpredictable. Kids running into the roads, chasing the ball. These tend to be some of the hardest cases.

All right, questions. And if not I have a ton. But over here, this is funny, just about the date. When are you going to see self.

Driving cars as a business on public streets in America?

Sorry?

When are you going to see self driving cars in the streets? Today. Already today to some degree.

You go to San Francisco today and you can get a ride around and enjoy it.

Do you want. No, go ahead and find your question. Part of the question was as a.

Business when we're going to see them commercially deployed.

And right now they're still in testing phase and I think they're not available to the general public yet. And some of them at night.

Yeah, yeah.

Driver inside. Some of them have no driver inside. That's rare.

Yeah.

But when you're going to see them as a business.

Yeah. So one of the reasons why Aurora is focusing on trucking is because the business we think makes more sense earlier that there's all kinds of really great social reasons. But you're asking about business. A truck driver, we pay about three times as much as we pay a ride hailing driver. We're short about 70,000 of them. So there's a real economic need and we expect that driving on freeways, it's much more similar than driving through different neighborhoods. And so we expect there to be operational scaling rather than technological scaling for us. We're looking to launch, we're working towards end of 24 where we'll have fully redundant trucks. At that point the trucks will be scalable and we'll expect to start to be building a really exciting and interesting business at that point.

Yeah, we're already doing commercial deliveries, but at small scale. If we are talking about more scaled commercial operation in multiple cities, I think this is going to happen in the next two to three years. That's for us.

I think this is all within a couple years. Another question and it's all right if no, because I will ask one. All right, I will ask the last question then.

So.

What do you think this plays to globally? The very natural thing is of course we have a whole bunch of intensive things playing here in the U.S. but there's a question about obviously there's a bunch of stuff going on in China. There's a bunch of different environments in whether it's India or Africa. So we've got all of this intense work working within the first World. How does this broaden to, as it were, crossing the digital dividend?

Yeah, that's a great, great question. I think there are probably going to be multiple systems. I think it's not going to be, you know, one winner take all. I think there for many, many reasons, one is just this is taking very, very long time to develop. It's very hard for the first one to have an advantage way bigger than the second one. I think there's also political reason, for example, the US and China why I think there's national security reason why US Company probably will be very hard for US companies to go map Chinese roads and then vice versa. But I don't think it's going to be 20 companies. I think it's probably going to be a very small number of companies in the US Maybe Europe, but maybe there are also a handful of companies in China. And then I think we'll grow into other countries.

And before we get to yours, because this to reflect is one of the points I'd learned from Sebastian years ago is that all of these vehicles out there are learning machines and they're sharing their learnings. So when you have it in the same network, even if it's in different geographies, you get an improvement from that. And that's part of the not n of 1, but small n will be the number.

Yeah, I also think it'll be small N. I think it's going to be Balkanized for the same reason Shajun does. I just we're seeing it on the Internet. This is going to be more scary to governments than moving data. Right. We're moving atoms through the world with a lot of energy. So I think that'll split China and a lot of the Western world. I expect it'll be kind of the US because of regulatory flexibility. Europe, possibly like Korea and Japan just because of the population demographics is going to be important there. And then it'll. I think two decades from now, maybe this looks like what happened with mobile networks instead of landlines. That is you are developing new cities. This is what you build the city around is automated transportation. And so that may actually interesting create a leapfrog there as basically the cost of the systems come down over time.

Yeah. One of the things that I think is also very interesting that we've all talked about is how much each of these vehicles becomes now a data center of inputting data. All of the things that could mean for how you make society much better. Like when you like, oh, I saw a person collapsed at the side of the road maybe we should call medical services. All of this stuff is part of the lens ahead on this. That part of why we at Greylock are so delighted to be partners with both of you is that you're thinking about these how does our vehicle massively improve society? What are our responsibilities on the downside, like safety? But also, how are people's lives so much better, including, like, work and truck drivers and all the rest? So, on the behalf of all of us, thank you for joining us.

Thank you. Thank you.

That concludes this episode of Gray Matter. If you like what you hear, please subscribe wherever you get your podcasts. And please let us know how we're doing by rating Gray Matter. We really appreciate your feedback. I'm Heather Mack, and thanks for listening.