Self-Driving Cars: Timothy B. Lee Answers All the Questions You Were Too Afraid to Ask

Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week we'll explore their revolutionary ideas, and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Thornberg. Hello and welcome back to The Cognitive Revolution. Today, I'm excited to share my conversation with Timothy B. Lee, noted autonomous vehicle industry analyst and author of the Understanding AI newsletter, online at understandingai.org. Tim has been covering the self-driving car industry since 2016, and his in-depth analysis of the various technology stacks, business models, and regulatory issues that have shaped the space make him the perfect person to answer all of my many questions on this topic. Over the next 2 hours, we characterize the current state of the art in autonomous vehicles, starting with our personal experiences with both Waymo and Tesla. We unpack the contrasting approaches of these industry leaders, including their different hardware stacks, data acquisition methodologies, and deployment strategies. We examine the technical hurdles that are still facing self-driving cars, from handling edge cases like emergency scenes to expanding beyond well-mapped areas. We interrogate company-published safety statistics and explain why it remains difficult to definitively prove that autonomous vehicles are indeed safer than human drivers, though it does seem like Waymo is getting close. And finally, we explore the regulatory environment, starting with the very notable fact that government oversight has not been nearly as restrictive as many might have feared. Along the way, Tim also explains why some previous predictions have proven overly optimistic and shares his current expectations on timelines for widespread adoption. We even explore the potential for new vehicle form factors once human drivers are no longer necessary. Throughout, Tim provides a very balanced view. He's clearly very excited about the fact that today's leaders are finally getting to the point where their products really work. Today, Tesla allows human drivers to take their hands off the steering wheel, and Waymo is beginning to scale truly driverless cars. And yet, at the same time, Tim is realistic about the fact that it probably will still take a few years before self-driving is very broadly available. For my part, driving is the most dangerous thing that I personally regularly do, and many people spend a lot of time doing it. So I really can't wait for AI driving systems to become the new normal. As always, if you're finding value in the show, we'd appreciate it if you take a moment to share it with friends or to leave a review on Apple Podcasts or Spotify. You can also DM me on your favorite social network, and we continue to invite your resumes if you're an aspiring AI engineer or AI advisor via our website, cognitiverevolution.ai. Now, I hope you enjoy this conversation about all things self-driving with Timothy B. Lee, author of understandingai.org. Timothy B. Lee, author of the Understanding AI newsletter at understandingai.org. Welcome to The Cognitive Revolution.

Thanks for having me on. I've been listening to the podcast pretty much since the beginning, so it's exciting to be part of it. Thank you.

That's flattering. And I'm excited about this as well. I've been a follower of your analysis of the self-driving car industry, both through your public analysis and through many one-on-one questions that you've been gracious enough to answer. So what I'm excited to do today is just try to lay out for listeners kind of beginning to end, like, where are we in this self-driving car journey? Ultimately, when do I get my car that I can sleep in on the way to my grandmother's house. And there's obviously a lot to cover between here and there. So, you know, kind of all angles is the goal and you're the man for the job. You want to start off with just a kind of simple overview of the levels? I think people will be familiar with the, at least have heard this before, but it's probably a good way to introduce. And I think you have a little bit of a contrarian take on it too. So you can give people your sense of what really matters in terms of the levels of capabilities in self-driving.

Sure. So there's these standard SAE levels, levels 1 through 5, and level 1 is just a regular car with maybe like regular cruise control. Level 2 is driver assistance. Level 4 is a full self-driving car like Waymo, but in a limited kind of area. And then you have level 3 is somewhere between those where it's like some kind of shared responsibility between the car and the driver. And level 5 is supposedly like a car that can go anywhere. But I don't find these levels to be super, super helpful because especially that level 3. Where it's never been totally clear. I guess the idea is a car where you sit in the driver's seat, but the car mostly drives itself. But then if it gets a problem, it hands over control to you. And that just seems like a kind of user experience nightmare and a safety nightmare because if the car suddenly has a problem and you haven't been paying attention, it's hard to be sure, are you gonna have the context to make the safe decision? What if you're turned around or you're putting your kid in a car seat or something? I think a much simpler way to think about it is there's 2 kinds of vehicles. There is driver assistance, which is any of the Tesla or any other automaker has. lane keeping, driver assist kind of things. And that's the system where the driver's still responsible. The car might help, the car might do a lot of work, but you are still supposed to have your hands on the wheel, eyes on the road. And then there is 4, like fully driverless vehicles where you're not on the steering wheel, the car is legally responsible. And so far at least, that always comes with a geofence where there's some places where it'll go and some places it won't go. I don't think that level 5 is really a useful concept where there's no geofence because there's always gonna be a service area. There's always gonna be countries or types of roads or types of off-roading that the vehicle doesn't do. I think about a little bit of cell phone service, right? To first approximation, like you could use your cell phone anywhere, right? But that's not really true. There's areas that don't have cell phone towers and nobody says, oh, that's not like a real cell phone because it doesn't, you know, go in rural Alaska or something. And so I think self-driving cars are similar for the foreseeable future. Vehicles that are driverless are going to have a service area. And the question would just be like, what is that service area rather than can this go anywhere or just some places?

It might be interesting as we get deeper into the technology stack and the different approaches that different companies have taken to try to understand better to what degree that limitation will be a matter of connectivity in the same way that it is with a cell phone or having a map or not having a map, versus if it's a self-contained system that's all on board on the car. Is it something where the car is just saying, sorry, this looks too crazy for me? To go any further.

I don't think it's just about connectivity. It's partly about the map, but it's also about things like regulatory questions. Like, in the future, I assume you'll need a license to operate a taxi service in certain areas. And also, if there's no driver and the car gets stuck and there's nobody in it, like, somebody's going to have to come service it. And so I think if you think about it as a service, as opposed to a, just a car, any company that offers a service has a certain set of areas where they offer the service and areas where they don't. And the car is only going to operate where the company has service. Now, there might be a future where the company literally has a footprint everywhere and so can pretty much offer service everywhere. But I think that's quite far away.

Let's maybe talk for a second about the 2 canonical companies that most people have experienced. I think still most listeners probably have not experienced either of these technologies. I think you and I are both in the lucky few, and I have to credit you for getting me a connection at Waymo that got me off the waitlist and allowed me to actually get into the Waymo a couple times. Last time I was in San Francisco. So thank you for that. The 2 are Tesla and Waymo, and we can get into other companies as well. But Tesla, it sounds like you put into the level 2 category. Last time I used it, you can add on to this with your own reflections on the experience, was for me about a year ago. It was last summer. I took an 8-hour round trip to actually take my grandmother home. It was both my and her first FSD experience together. And I would say it was very strict about enforcing that you are paying attention. If I understand correctly, it has a camera on you from the rearview mirror that's monitoring to make sure you're alert and engaged. And you also had to have hand on the wheel with a little bit of torque, which was something it took a little getting used to for me in and of itself. Otherwise, it would say you're not paying attention and execute like a pullover and shutdown of the— self-driving experience. So that was like pretty short leash on the human in the seat. I did find the experience to be overall very good. Like, I came away extremely impressed. I probably let it go a little too fast on the highway, honestly. My neighbor who lent me the car had a default setting of 20% over the speed limit, which I just naively followed. And then when I got back, he's, oh no, I turn that thing up and down all the time. It just depends on where we're going. So I was going 84. In a 70 at night in the rain on the way home, which, as I look back, I was like, maybe that wasn't so smart, but I was definitely paying close attention. And overall it did really well, certainly at those high speeds. I thought it was amazingly good when it was— the only kind of areas where I found that it messed up a bit was in ambiguous situations, like where there's a stop sign that could be interpreted as facing one of two ways. And I I've personally been a little bit confused by some of those situations, too. That was one where it was probably the most risky thing that it got wrong, was stopping on the off-ramp of a highway where it definitely should not have. But overall, I thought that was really good. I think you've used it more recently than I have. The hands are now free. What else is kind of new in that experience? And is it still 2 at that? Would you still put that? I know you're not a lover of the levels.

This is why I emphasise liability. I think the fundamental question is, if a car crashes, you get to sue the company, or is it Whoever you hit, sue you. Because ultimately there's a technological design decision of how strict is it going to be about enforcing the requirement to do it. But even if the car is very advanced in the Tesla system, they're still going to blame you if you crash the car. They're going to say you should have been paying attention. And so that's the distinction I would draw there. So my experience was back in March. I spent about 45 minutes behind the wheel in San Francisco. And so I was not testing the strictness of the driver monitoring because San Francisco is a pretty hair-raising place to drive. I was happy to do it in San Francisco, both because it gives it a good challenge for the vehicle, it also because that's also where I'd taken some Waymo test drives. So it's a little bit of an apples-to-apples comparison. So I did not try taking my hands off the wheel or taking my eyes off the road because I was like, wanted to make sure I didn't crash the vehicle I was borrowing. So I don't know about the driver monitoring, but my experience I had, like I said, I did it for about 45 minutes. There were 2 times when I took over during that 45-minute period. There was one time I was making a turn onto a road that had a bike lane with some plastic separators, and it was about to hit one of those. plastic bollards or whatever you call it. And the other time it got in the wrong lane and then was trying to cut over into a lane that was full. And so there are cars behind me honking because I was like, so that wasn't technically a safety issue, but it was definitely a mistake and something that I was annoying people on the road. Yeah. So that was my most recent experience with it.

And was this, do you know what version that was? We're now on 12.5. Okay. Yeah. So this was after the big Streamlining to neural networks, but before the very latest.

Exactly.

Okay. On the Waymo side, I think what people most envision when they dream about a self-driving car future, and it is amazing that it's here today, it is in a limited geographical area. Like, I wanted to take it to the Oakland Airport as I was leaving town. Can't do that. Can't cross the bridge. Can't get out of the San Francisco city limits. But It was within the city, really an amazing experience. I mean, you use the app. The app is basically an Uber-like app. I have some questions about what this may mean for the future of Uber as well.

Yeah.

Summon the car, pretty quick response time. They seem to have liquidity in that San Francisco service area. It pulls up, it pulls over. I think it shows your name on the top of the thing, which is cool. Honestly, probably the hardest part of that experience was figuring out, at least as a first timer, that I needed to unlock the door in the app before I could get into the car. That was maybe a little UX refinement still to be done there. But then once you're in the car, it's wow, this thing, there's nobody in the driver's seat. There's just a sign there that says, don't mess with the controls. And you ride along and it takes you where you want to go. And I think if there was anything that was most striking about it, It was how easy or how quickly I adjusted to it and started to become bored. Like, I'm somebody who's very enthused about this technology. I was like, this is a privilege to be in here. But then we'd stop at a red light. I'd start to look at my phone and next thing we're going again. I'm still looking at my phone and I'm like, wait a second, get your eyes off the phone. Like, actually pay attention to what's happening. It was just amazingly comfortable, normal. And in my not super high experience, but probably an hour over a couple trips. About as close to flawless as I could say. I would say it was a more comfortable ride than an average Uber, maybe not quite at the level of skill of the very most skilled drivers who might have made that extra light or whatever. But also, that's not exactly what they're trying to get. They're not emphasizing making the light. So I was super impressed. I know you're super impressed with Waymo too. What would you add to that?

Yeah, so I've done about 3 hours of driving over a couple trips to San Francisco this year. And similar experience. So I, over those 9 rides, I never saw it make it, you know, even like a minor mistake. Definitely nothing, no safety issues, but also just the ride was very smooth. It never got confused or got in the wrong lane or anything like that. And yeah, like you, it was first couple of rides I was paying close attention. I think I took a video of the first one. But by the 5th or 6th ride, I was like reading a book or looking at my phone. It's just, it's, it's just when they're that consistently good, it's just not interesting. Like nothing.

Yeah.

Nothing interesting is happening. So, yeah, I'm very impressed overall with the quality of the driving.

So we've got these kind of 2 apparent leaders, and they have quite different strategies in a number of ways. With the highest level, it's like Tesla is a car that you own. It is available, quote unquote, everywhere. You can refine that, but it's nationwide anyway. And you still have to be engaged. And there are some mistakes that we've each seen that we've had to catch. Although I would never say I felt radically unsafe. Like, I definitely did have to do stuff. On the Waymo side, it's a robotaxi. It is very narrow footprint, but it seems to be more polished, at least within its kind of designated surface area. So Tesla has to close the gap on whatever that last Probably even less than a 10th of a percent now. And Waymo has to expand its footprint to cover everybody if they really want to bring in the dream of the self-driving future.

Yeah.

What makes this hard? Is it the same thing for— people obviously have a sense that, wow, there's a big world, there's a lot of random things can happen. Is it the same kind of hard for each of them still, or is it a sort of different fundamental challenge that they each have to solve? to really get over that hump at this point?

Maybe this shows my bias, but I think of there being sort of different stages. So I think some history is useful here. So I've been covering this industry since 2016, 2017. And back then, Waymo was just this Google self-driving car project. And they had to file annual reports about their disengagements with the California regulators. And in 2017, February 2017, they reported that during 2016, their vehicles, their safety drivers had to take over once every 5,000 miles. Now, there's probably some cherry picking. They would go back and look at them and say, this is a really cause of safety issue. So you can like question that. But way back then they were, I think, in a similar position to where Tesla is now, where the cars pretty much always worked. There was occasional kind of weird edge cases. And they really felt like they were ready to do the Darvellous thing. And they put in orders for 60,000 vehicles in 2018 from, yeah, Chrysler and from Jaguar. And they thought, I think they sincerely thought that they were on the verge of having this nationwide taxi service, and then it turned out to be much harder than they thought. And the service they launched in 2018 ended up having safety drivers for another 2 years before they went driverless. And it was very small scale in 2020, 2021. And it was only a year or 2 ago that they really started to ramp up in scale. And it's not totally clear to me why, but I think it's on some level, it's just about their— the edge cases. Some of them are really hard.

Yeah.

And the one that I've been thinking about the most recently, and that I think you've seen a lot of news reporting on in San Francisco last year, is emergency scenes, either a car crash or a fire scene, where the stuff that happens there is just very different from the typical situation. So in a normal situation, there are lanes, the car has to stay in the lane, there's rules about you turn here and you don't turn here, you stop at this light and so forth. When you get to, to a car crash, a police officer will come and they'll start directing traffic in a totally different way. And every car behaves differently. And so then you have to reason about, for this car to get, go here, I have to stop or I have to back up or this police officer wants, there's just all these, it's just a totally different problem. And that might only happen once every 10,000, 100,000 miles. But if you have a, and if you have a safety driver, it's not a big deal because the safety driver can take over if the car gets confused. But if you have a driverless vehicle, the driverless vehicle has to handle that somehow. And I think there's just enough of those situations. That if you have a driverless vehicle that only does 99.9% of the time, it's still getting stuck all the time, or maybe even running into people sometimes. And that's just not good enough. And so the last 5 years has been Waymo going from 99.9% to what they are now, which I don't— it's a little hard to tell exactly what the kind of number is now, but now they handle enough of those edge cases that it seems to be a kind of viable service or close to it. And I think Tesla's just at the start of that state, that process. My guess is that what if a Tesla comes up to a scene of a car crash, it'll just signal the— it'll either get stuck or it'll signal the driver to take over. I've never tried it myself, but I assume it's not ready to do that the way a Waymo does pretty well.

Hey, we'll continue our interview in a moment after a word from our sponsors. I don't have any direct info on what a Tesla does in that situation either. I did ask my neighbor who drives his a lot, or has it drive him a lot, and he said Yeah, it doesn't really take sort of unconventional moves to handle strange situations. One I specifically asked about was like, if somebody is coming up on you and appears like they might be about to rear-end you, would the car move forward to give the car that's coming in hot more space? And he was like, no, I don't think so. It's probably not going to be that creative in a situation like that.

Yeah.

I guess there's, is there, is this one thing or is it two things? The, my instinct as somebody who loves to test AI products in all different formats around the hand signals thing would be like, I sort of bet that like the latest Gemini model, then we're talking on the first full day when Gemini tops the LMSYS leaderboard with their latest August 1st release.

Mm-hmm.

And it can handle multimodal. You can put video into it. My guess is that it would, out of the box, be pretty able to interpret a scene like that and say, what does the officer appear to be signalling in this situation? Obviously, you have to be more reliable than probably Gemini is going to be. But is it just ultimately just the volume of these unfamiliar— I guess my gut is saying that still feels common enough.

Yeah.

That it doesn't seem like the kind of thing that should be that far. If it can do all these things that it can already do, why won't it be able to learn the hand signals? Or are the hand signals really that never seen before that you couldn't expect similar performance there?

I think there's a couple of things. The main thing is just it's different from what the main training data is. So the fact that Tesla can stay in your lane on the highway just doesn't tell you much about whether it's going to do those other things. It's pretty rare. So it's rare enough that you're going to need a lot of miles. before you have training data. If you're going to do a kind of learned approach, you're going to need a lot of training data. Now, you could take a car on a test track and have somebody pretend to be a police officer, but it's hard to think to do it realistically. A real crash scene has dozens and dozens of cars in kind of random configurations. It would be expensive and hard to do that in kind of a fake situation. But then I also think the kind of reasoning is pretty different. When you're on a regular road, there's a rule you can learn that says you always stay in this lane. Whereas in a crash situation, you really have to reason counterfactually. You have to anticipate what are the other drivers going to do to a much greater extent. Like, this vehicle's trying to get out of this lane, and so I'd better not get in the rain because then we'll have a— so I'm definitely not saying it's impossible for AI systems to do it. I mean, I think it seems like Waymo is getting there, and I'm sure Tesla will get there eventually. Oh, and one of the things, I think that with Tesla's current approach, I think that most Tesla owners' instincts will be to switch off FSD when, as they're approaching it, which means you're not necessarily going to get good training data. Like ideally what Tesla would like is for the human driver to allow the FSD to try it until the point where it actually makes a mistake and then take over because then you have data on this is, you know, how it works in a real situation. But if the human driver is like worried about annoying the cop or whatever, then just turns it off before it gets there, then you don't have as much, you don't have good training data. So Yeah, I'm not saying it's impossible. It's just empirically, from looking what's happened with Waymo, it seems like a much harder problem than the other. I don't think the people at Waymo and Cruise are idiots. I think they sort of knew this was a problem. I'm sure they made some effort to prepare for it, but it turns out to be just empirically watching how it's gone for them. It seems to be a very difficult thing to deal with.

Yeah, that's a good transition to one dimension. And I want to go dimension by dimension here of how these companies have approached this problem, what their strategies have been, the strengths and weaknesses of those. This is a good entry point to data acquisition and I guess overall learning strategy. The sort of Tesla bull case, as I understand it, is they've got by far the biggest deployed base. And I understand that they collect data on human driving. So when you say if the person is a little bit nervous or whatever and just takes over, Would that mean that Tesla wouldn't get that data? Because my naive understanding would just be like, if they can continue to record what the human does, then isn't that sort of, in a sense, like labeled data or not gold standard, but there's some sort of ground truth there of like how the human navigated that situation. So am I misunderstanding? Do they not collect as much as I thought they did?

So I don't think we know exactly how they do it. I don't necessarily think that— I think it's quite possible they aren't able to collect the data as a human being is driving. I mean, I think it's just less useful. You can try to go back in simulation and simulate what our vehicle would have done in this case, but because it's an interactive kind of thing, like it's not often obvious in the moment. The question is not, did our car hit the other car? It's, would this car's decision have led to a situation 3 seconds later where everybody's jammed up because the car went the wrong way and got in somebody else's way? Stuff like that. That data is, I think that training data is definitely useful, but it's not I think, as useful as if you had a professional safety driver who was a little more aggressive about letting the vehicle do its thing until it got into a mistake. But yeah, absolutely. So Tesla does have the ability to gather large amounts of data from its vehicles, from its fleet. And as I understand it, it has some pretty sophisticated tools where it can say, okay, we're trying to train on cars with bicycles strapped to the back, and they can send some kind of query out where it says, we're looking for this kind of image. And then the car will run some sort of local search through its kind of database and send back candidate images. And so they have a very good ability. I do think this is one of the advantages Tesla has. They have a good ability to both collect a lot of data and also focus a little bit on, here's the kind of data that we're looking for. And yeah, I think that is plausibly one of the advantages they have over other companies that are trying to do this.

They would get data from all cars or all cars that haven't opted out? You don't need to be an FSD customer to get to be relaying back camera data to—

I looked at this closely, but yeah, I don't think it's limited to FSD. I think anybody, I think by default, probably when your car is close to a WiFi point, the Tesla can suck down images or telemetry from recent trips you've taken.

Yeah, I would assume the Elon terms of service are pretty friendly in terms of sharing data back with the company.

Yeah.

Okay, so there's like a— would seem to be a big Data advantage, but I guess to the degree, or the degree to which that advantage matters probably depends a lot on like other aspects of the technology stack. So when I'm hearing your analysis of would the car have done something which 3 seconds later would create a challenge, that opens up a number of questions for me. I guess the first one is, what is the cycle time at which these things are running and making adjustments? Because if it is multi-second, then it's, yeah, you, you're doing a, a significant amount of— modelling and predicting what other people are going to do in response, it seems to get complicated. I could also imagine a stack where the sort of next move, the next token time, if you will. I'm used to language models, right? So these things are measured in tokens per second, and now we're often getting into hundreds of tokens a second. If you could run an inference cycle at that same frequency, then it would seem like you could maybe be a little less worried about the butterfly effect of if I did something slightly different, what would happen downstream? And more just focused on imitation learning of, can I learn to do exactly what the human is doing on like a hundredth of a second or a tenth of a second sort of basis? So I guess first question there is, you could maybe just zoom out if you want and take this from the beginning, but the frequency is what I'm stuck on at the moment.

So let me walk through the standard architecture. So I think of it as 3 layers. So there's perception, layer, which is the layer that takes the sensor data and builds a world model. It says, here's all the objects that I can see with my sensors, their locations, their velocities, the types of objects they are. So that's the first layer. Then there's a prediction layer, which says, given all this data I have, what is the world going to look like 1, 2, 3 seconds ahead? And that's important, obviously, because if there's a car coming from the opposite way in an intersection and it's not currently in front of you, but it's a certain velocity and it's going to be in front of you, you need to be like— Playing that out. And then a third is planning, where given this set of current states and future states, what's the path that gets me where I want to go without causing any collisions, et cetera? And so I think that just a very naive, short-term imitation learning approach, I think, is not really going to work because what the human is doing is the human has a prediction engine in their head, right? You're thinking about in 3 seconds, this car is going to be in this location. And so if I'm also in this location, there'll be a crash. And in the same way that language models, when you train a large language model, it builds, it learns higher-level concepts that then allow it to predict the next token in a way that's maybe not explicitly planning what the future tokens are, but it's able to do higher-level reasoning. If you had a model that was able to learn the implicit model that humans have, I think that could work in theory. But I think what, and actually I think that it's evolving in that direction. So 5 to 10 years ago, it really was true that Like the Waymo and Cruise stacks were a bunch of pretty hand-coded rules where if you're in this kind of situation, do this, that kind of thing. And over time, I think they have replaced more and more of that with neural networks that learn. But I think it's still probably a single-digit number of— there's probably one or a few perception networks, there's probably one or a few planning networks. So they publish papers about a few of these. So, yeah, so I think the trend has been towards more learn neural network, deep learning kind of things, but it's not They're not yet to the point where there's just one big neural network that just has sensor data and pedal and steering wheel positions out. And I'm not sure that they're going to get there or that it would make sense to get there because you do want to have— you want a much higher level of reliability than you see with a large language model. And I think there's a lot you can do. You at least want to have visibility into the vehicle's implicit knowledge model of the world so you can verify, yes, it's seeing the things that really are there, and yes, it's making reasonable predictions. I think some of the training you can do You can train the perception and planning and prediction layers separately because you have ground truth there. You say, I actually know where this car was, and so I can see, was my prediction of where the car would go accurate? So yeah, I think that's the direction it's headed. It's going to be a small number of networks, but not one. Tesla has talked about having an end-to-end network, and I don't think they've been super transparent about exactly what that looks like. I'm a little skeptical that it's literally just one end-to-end network that takes sensor data in and outputs a driving direction. But I'm not sure exactly.

Hey, we'll continue our interview in a moment after a word from our sponsors. It seems there is at least a— they display for you in both of these examples, they display for you on the dash a world model. So you can see that there is something else happening besides just straight to actuation on the controls. Those are both striking. It's like an incredible thing to watch unto itself. Honestly, if you have never sat in one of these cars, even if you just forgot about the fact that it can drive itself for a second, and you just looked at how well it is understanding the scene around you and labelling those things and knowing what's a person and what's a vehicle, and that in and of itself is just pretty impressive. We just did an episode not long ago with 2 authors, Nora and Ben, of a paper called Guaranteed Safe AI. Which is a proposal for a framework that seems like it's almost perfect for self-driving. First of all, it kind of has 4 components, but 3 are added onto the main AI. So their idea is maybe you have a black box AI, maybe you train this thing to be 100% end-to-end. We want to be able to audit this. We want to be able to have a much better sense of what's going on than we can get by just brute force poking at it with situational testing. So how do we do that? The 3 components that they add on are a world model. So it seems pretty clear that these cars have some version of that.

Yeah.

Then they advocate for an explicit safety specification. I don't know if the makers have anything like this today, but they're envisioning things there like acceleration should never exceed whatever, or we should never get closer to another object than whatever. And to make that clear, one of the benefits that they see on this is that it enables a societal conversation about what the safety standards should be. And then the third piece is a verifier, which is the thing that takes the output of the core black box AI and simulates what will happen if we do that according to the world model. And does that still fall within the Safety standards. It seems like that's what's going on here. Does that sound like the architecture that these companies already have?

Yeah, at a high level, that's what they're trying to do. I assume that Waymo has internally, either explicitly or implicitly, rules like that. Don't exceed this acceleration. Don't get any closer than X to a car. One of the things I think is difficult, so Mobileye has a standard called the Responsible Sensitive Safety standard that tries to do this, to say, here's our formal specification of how we think a self-driving car should behave. And I think that's a perfectly nice idea. I think that one of the things that makes it really difficult is that you're not just trying to optimise for safety, you're also trying to optimise for getting where you're going and not irritating people. So one way you can make a vehicle safe is have it be very conservative, and anytime it's not sure what's happening, it slows down and stops. And at least if you're not on a freeway, that's always safe in the sense that nobody's going to die from that. But if you have a vehicle that's super overcautious, it's going to cause traffic jams and piss people off. And also, if you have a fire engine coming, then you could indirectly cause safety problems. And so I think part of what makes this problem challenging is you want to never get into a crash, but also behave like a normal driver. And when it is safe to go, you want to go. And I think that trade-off, I think it would be much harder to capture in a simple some kind of standard. It's just there's lots and lots of different iterations. And ultimately, I think you just need a tonne of examples, a tonne of training data, and that's something you need to learn. And yeah, I'm all for more transparency, but I think it's like a hard problem to formalise it.

Yeah, I think the way that they would respond to that, and I can't speak authoritatively on this framework, but I think the idea there would be that the core AI that you've trained, that this sort of guaranteed safe framework sits around can still be responsible for the sort of human driver experience and the not annoying people and all the fine points. And it's okay if those remain opaque in terms of exactly how they're working. It could be a little bit more permissive than— and in fact, almost by definition, you need it to be a little bit more permissive than what the system actually does. But you could hopefully get it down to a finite number of, and hopefully reasonably understandable number of rules of the thou shalt nots of this problem, which obeying all the thou shalt nots is not sufficient to have a great user experience, but it would hopefully at least be sufficient to say we can make some quantitative guarantees around very bad things not happening.

Yeah, my guess is that Waymo would pass that fairly easily. I mean, the other issue is that Sometimes you have planning mistakes, but sometimes you have perception mistakes where you think that the child is a garbage can or something. And I don't think there's any way you can formally prove that a car won't misunderstand its models. Yeah, I think you could probably apply a model like that to the prediction and planning parts, or at least the planning part, and say, given a particular world state, we can prove that it won't make a mistake relative to the specification. But that's only going to capture a slice of the possible errors it could make. But yeah, I would be happy to see companies do something like that as part of the safety case they make to the public.

So going back to the frequency thing and the sort of planning, this maybe is just not disclosed, but I guess my experience in, I would say, both of these cars, but the Waymo One is most fresh in my mind, was that it did feel like the response time was human-like. It felt like it was responding as fast as I would respond. I actually don't know what my own response time is. Maybe you have a stat on that would be like the human point of comparison. But do we know how far these things are planning and predicting out? Like, how far does their world model extrapolation go? And do we have any sense for how often they are updating their outputs to gas brake steering?

I don't think they've disclosed anything like that, and I bet it's complicated. I remember maybe a year ago you had a drone maker on where they talked about there being different kind of levels of abstraction where there's a kind of high-level navigational thing that maybe updates every second or something. And then there's like a real-time make sure the drone stays stable thing that's operating at like a millisecond level.

Mm-hmm.

My guess is it would be a similar thing where there's some systems, the system that kind of figures out what's our overall direction, that's probably pretty slow, but then Somebody runs into the car, that loop that's, let's slam on the brakes, maybe is very fast. But yeah, I don't think they publish any specific data on which things happen at which frequencies.

Yeah, that was a— deep in the catalog there. I appreciate that. That was July 4th last year, Haik Martiros from Skydio. And I learned a lot from that episode just in terms of how these sort of different frequency and different level of abstraction control loops interact with each other. I think they had at least 4, maybe 5. The very lowest level one was, I think, like tens of thousands of hertz. And that was just at the level of the voltage being applied to the motor could change that fast. And then, yeah, at the outer layer, it was like language model instructions of planning, how am I going to go survey this bridge or this building or what have you?

Yeah.

And connecting all those things where each level interfaces with the one above and below it was a huge part of what they had invested. And they're really just adding on the last layer now. Interestingly for them, I think the— and this may be quite different from what I'm gathering from the self-drivers, but pretty much all of the lower level stuff was either explicit code, like they'd done a lot in explicit code, or very small neural networks, and that was all there. And now with this new generation of AI, they were able to layer on one more layer at the top where the user could give high-level instructions. But a lot of that stuff below was older technology and had been built out largely in explicit fashion over a long time.

Yeah.

So interestingly, a great callback. Okay, so we don't know ultimately how fast they can respond. I guess I can only say it felt like I wasn't noticing anything faster, or I wasn't feeling any lag relative to what I would have done. It's like a pretty qualitative statement. But maybe let's do a deeper unpacking of some of these technology stack decisions that folks have made. We've covered the 2 form factors that you can get the car in. You can buy it, or you can go on an app with Waymo. One other one that is maybe worth considering, or maybe you think it's a red herring, is the aftermarket kit, like the retrofit style, whether that's Comma perhaps or some other thing. Do you see that being a meaningful form factor that we should also be thinking about?

I think Comma is a very interesting company. They have a very active open source community and they've managed to build or add support for a tonne of cars. So lots and lots of, if you have a reasonably new car, There's a good chance you can apply Comma to it. And Comma is— they sell a device that's just like a little smartphone, I think, basically with a camera on it. And you plug it into— there's a— every car has a port that gives you access to the car's onboard network, and you plug it in and it gives you kind of Tesla Autopilot style level 2 self-driving. I don't expect that to be like a big market because for one thing, it's been a regulatory gray zone. Comma, I think the fact that the user downloads the software separately and installs it themselves, I think may give them a little bit of They've had some friction with NHTSA, the federal regulator, about this in the past. So partly for regulatory reasons, but also just because I don't think most people want to screw around with open source software. So I could imagine if— I think the bull case for Comma is maybe at some point it will license this technology to carmakers to build Comma on. But I think there's lots of good reasons to have that be vertically integrated, ultimately have your car manufacturer pick a good system and put it on there for you.

Yeah, it definitely takes the notion of AI hobbyist to the next level when you're strapping it onto your Chrysler minivan that's however many years old.

The other thing is, I think that the carmaker can integrate the sensors much better. So it's impressive that Comma can do what it does with basically one smartphone camera, like mounted, like on your windshield. But in the long run, you're going to have— I think Tesla has 8 or 12 or something cameras in there. I think in the long run, that's what you're going to want. And you might as well have software that's integrated with that specific sensor suite as opposed to having some aftermarket thing that's not going to work as well with the specific characteristics of your vehicle.

Yeah, it's amazing that they can do much of anything with just one point of view. Does Comma also suppose or require that the car itself has additional cameras?

No, I don't think so.

It's really just using the—

There might be a rear sensor, but yeah, it's like much fewer sensors than a Tesla has. And yeah, I think it's just based on the camera that's in the little device you put on your windshield. So it's pretty impressive.

Yeah, cool. Okay, then let's talk hardware maybe for the other leading strategies. Tesla is famous for an all-camera approach. I don't know if they're still strictly on that. One look at the Waymo and you can tell that this is doing something a little bit differently. So break down for us how people are sensing the world.

Yeah. So the 3 major sensors people talk about are camera, LiDAR, and radar. Teslas used to have radar, and I believe about a year or 2 ago, Tesla took away the radar. So they're really committed to this camera-only approach, whereas Waymo has all 3. Waymo has a LiDAR sensor on the top that's a big long range. And then it has, I forget the exact number, but maybe 4 or 6 LiDARs around the edges of the vehicle that are short, shorter range, wider angle. And I think that's partly for if a pet or a kid were right by the vehicle, like it wants to know that. And then it has a bunch of cameras and I think, I don't know how many radars, but several radars. So they have 20 or 30 sensors on the vehicle. Whereas Tesla, I think, again, I think it's 8 or 12, something like that. Cameras and older vehicles had one radar. And these sensor modalities, I think, have different strengths and weaknesses. Like one thing radar does that's good is it can tell velocity is in addition, it can like directly measure velocity in addition to kind of position, which can be useful on a freeway. There are some LiDARs that do that, but most of the LiDARs people are using do not have that kind of velocity sensing ability. LiDAR has the advantage that it is high enough resolution that it can get a pretty good 3D structure for things and it can directly sense. LiDAR bounces a laser off the object and then measures the time to return, which means it can do it exactly. exact distance measurement, whereas camera of course just gives you an image and then you have to do post-processing to figure out what's the 3D structure of this 2D structure of the thing you're doing. And so LiDAR gives you a 3D point cloud that is different. And anytime you're doing— anytime you're trying to understand the world, it's useful to have diverse types of datasets because they tell you different things and they're going to have different failure modes. So yeah, that's why Waymo does it that way.

So I think I've seen online analysis that I think you may disagree with that basically says the camera approach is the winner. Like, cameras are cheap, they're commodities. The LiDAR is like more expensive, takes more maintenance. But my sense is you don't think that's actually as big of a deal as commonly argued.

It's definitely true that LiDARs cost money and take work to install and maintain. If it were the case that you could get very high quality with cameras, that would definitely be preferable. I guess I think it's just important to think about it in engineering terms. The goal here is to make something very safe, and nobody has managed to make something— we'll see. At least we don't have clear evidence yet that anybody has managed to make something that's very clearly safe. And so if you're a company deciding, there's a kind of safety versus cost trade-off, and you can pick different points along that. And Waymo has chosen to go with the higher cost, higher safety realm. There are some people who claim it's somehow like a disadvantage to have LIDAR because The camera only, for some reason, like cameras let you use deep learning more or something like that. And I think that's just a misunderstanding of the technology. There's no reason you can't have a neural network that uses LiDAR as well as camera. The big thing that I think, the big advantage LiDAR has, as I was saying before, is it can do direct distance measurements. So you can create a neural network that has, takes multiple images and do parallax effect, compare kind of the angles of things between different images and do a pretty good estimate of how far things away are. And so you can generate a point cloud that kind of looks like a LiDAR point cloud. The difficulty with that is that if that algorithm misunderstands what it's seeing, it thinks something is an object that isn't or misses the existence of an object, then it will get a wrong value for that. And the problem with that is that is, I think, can be correlated with other failure modes of vision. And so in the average case, it's going to work almost as well. But what you really care about is these edge cases where something weird happens, where there's an object your stack hasn't seen before. And LIDAR will always tell you there is something so many metres ahead that's a size. And if I were the CEO of a self-driving company, I would be really nervous about doing the camera thing because it might work 99% of the time. And then the 1%, when it fails, then my company gets a $10 million lawsuit. I think Waymo's approach works. It makes more sense, but it is an empirical question. If Tesla continues improving rapidly, maybe in 2 or 3 years, like, it'll be— showed that you can do it with cameras. I'm not super dogmatic about this, but they aren't there yet. And so I don't think that they've kind of made their case yet.

So, yeah, it seems like this may tie to a couple other aspects of the overall strategy that these companies are taking. Would it be fair to say that the LIDAR strategy is a balancing force that might serve to even out the data scale advantage that Tesla has? I guess what I'm thinking there is, like, Tesla's got a lot more cars on the road. It's collecting presumably a lot more episodes or whatever. But if those inputs are fundamentally noisier, then maybe you just need a lot more to ultimately learn what you need to learn. Whereas if you have much more precise measurements of exactly what the structure of something is, exactly how far away it is, or exactly how fast it's moving, which it sounds like the Waymo is getting all of those all the time, Then maybe if that's just a lot more accurate, then you have a less— obviously everybody still needs a lot of data. I could see that if it's that much more precise, I could see that, hey, maybe you could get away with an order of magnitude or maybe even 2 orders of magnitude less data. Does that seem like a reasonable way to think about those trade-offs?

I think that's probably true that Waymo has less data, but it's probably higher quality, both because they have LiDAR and they also because they have professional safety drivers who are annotating why they disengaged. They get a clearer signal for the, you have a reinforcement learning kind of thing where you have a clearer signal about this was a mistake by the car versus this was a time when the driver just needed a bathroom break, that kind of thing. So yeah, I would say in general, Waymo has less data, but I would expect it to be higher quality. But I think the main thing that's going on is just that Tesla's business model does not allow them to put a $10,000 LiDAR in every car. And so they have to do the thing they're doing, and then they're doing their best to make that work from a technology perspective. And maybe it will work or maybe it won't, but Waymo's strategy. is they're vertically integrated and they have a fairly high margin business. Like, tax, like, if they can get the— I guess we'll talk about the economics later, but if they can get it working, it's potentially pretty profitable. And so they can just afford to buy— they've only got a few hundred cars on the road so far, so it's not that expensive to put pretty expensive LIDAR on every vehicle. And the bet is that over time it'll get cheaper, and so then it won't be a big deal once they're at scale.

Yeah. So why, going back to the Tesla can't do it, if I understand that What you just said correctly, the case there is like at scale, a $10,000 LIDAR maybe becomes a $1,000 LIDAR. Maybe that could work for Tesla. They're already not the cheapest cars on the road.

Yeah, no, absolutely. But like when in 2016, when Tesla started selling what they characterized as full self-driving ready thing, back then the LIDAR cost, I don't know, $50,000 or something. Then also they don't, they put the cameras on every vehicle, but only 10 or 20 or 30 or something percent of people. Sign up for it. And so that business model, you can't, it just would be way, would've been way too expensive to do it back then. At some point, yeah. Yeah.

You can't sell that over the air.

Yeah. Once LiDAR gets scaled up and gets cheap, and if it's a $1,000 sensor, then yeah, maybe they will start putting it. Now Elon's pretty dug in, so I don't think he would want to admit that he was wrong. But yeah, if, if at some point LiDAR will get pretty cheap and I, I expect actually what we're starting to see, I don't think this has happened yet, but we're starting to see OEMs planning to put LiDAR. Other car OEMs plan to put LiDAR on their vehicles. I think you'll start seeing LiDAR be a standard kind of component, like radars in a lot of cars. And eventually I would expect Tesla to add it. But yeah, so far I don't think it's gotten cheap enough that it would make sense. There's also different levels of kind of quality of LiDAR. So the big spinning LiDAR that's on the top of the— well, that's still thousands of dollars. There are, I think, some other ones that are hundreds of dollars that have less field of view, less range, et cetera. And so those, I think, You probably could put in a vehicle, but probably are not powerful enough to add the kind of value that the big LiDAR on the Waymo adds.

Are they hard to maintain? I've also heard that this is an issue where like the Waymos are going into the shop, you know, regularly for LiDAR maintenance, even if it's all good otherwise. How big of an issue do you think that is?

That I don't know. Waymo makes their own LiDAR, and so it's pretty secretive. There's different— in the broader market, there's different models. There's some that are more solid state than others. Yeah, I don't know the details, but I think it's like anything else. It's probably somewhat hard to maintain now, but as they improve the technology, it shouldn't be an issue at scale.

It's interesting how secretive this stuff is. It feels like this would be the sort of business where it's hard to copy, even if you give some details of how you're doing it. So given that, and given the critical role of trust in a system like this, Am I off base for thinking that maybe they should be a little bit more transparent about what is actually happening?

As a reporter, I would love that. I think they've been reasonably transparent by the standards of technology companies. Google's not telling people what's happening in their data centres. They have published a number of technical papers, machine learning papers, about the various algorithms they use. There was one a couple of years ago where they're using transformers to predict the behaviour of vehicles on the road, stuff like that. And then they do have a safety report they put out, or they put several safety reports with crash data and stuff like that. So I think they've published a lot of the information that you'd want to evaluate safety. I think they've been an okay citizen on that. I would certainly like them to be more transparent about in a number of areas, but I don't think they've been notably secretive. And Tesla's a similar story. They've had a couple of AI Days or Self-Driving Days where they've provided a lot of detail. They've given talks about how various things work. And so I think we have a kind of high-level understanding, but the technical details are both some proprietary, but also I think they're changing pretty fast. And so if you look at something that somebody said 2 or 3 years ago, it might just not be the same as what's happening now.

Yeah, that's for sure. You can't go back and read the GPT-3 paper and expect to be making good inferences about what's going on with the latest models. So I would assume there's been quite a lot of change. It seems like the big thing has been The overall story in almost all of AI is toward more and more stuff being trained into the neural nets. Like, this is classic Elon. Best part is no part. Relative to early versions, I would guess there's probably been an order of magnitude reduction in just how many sort of components there are in the overall software. Is that your sense too? Do we know what are the sort of explicit things that remain. I guess you could think of that as sort of the safety spec or some sort of governance perhaps, but it could be more than that too. That's probably the kind of thing they haven't really shared. For example, how explicit do we think that planning still is? I recall the one language model for lane changes or sort of intersection where lanes can get weird. Tesla showed a module where they had essentially used a language model-like structure to figure out how How does 3 lanes become 2 as you pass through an intersection and whatnot? Previously, presumably, that would have been done with a bunch of rules. And so I guess I'm wondering, do we know what pockets of these hardcoded rules remain today?

I definitely think the trend has been towards more learned systems, less hardcoded rules. Waymo's approach has been to publish academic papers sporadically and not be super specific about the overall architecture. I don't remember off the top of my head how recently they published something about their planning. Stack, but they'll say, like, here's a network we're using that does some aspect of it. But I don't know that they've ever published kind of an overall paper that says, here's our overall architecture and all the networks they're using and how they fit together. It's more, I think, when they have something that's interesting and I guess not so sensitive that they don't mind really revealing it, then they will publish a paper. Tesla is a little different in that they are more presentation oriented. And so there's been, like I said, 2 or 3 or 4 presentations where Elon and a few of his kind of top AI people will give an hour-long talk saying, showing demos of various things they're doing and explain a little bit how the technology works. But again, I don't think they've ever given like a whole kind of architectural overview of all the networks or all the software modules that make up the system.

Tesla once famously open sourced all their patents. Do you know what their status is on that today? Are they just not patenting this stuff or are they no longer open sourcing all patents?

I've not looked into that. I think that it's normal for companies to patent a bunch of stuff and then mostly not sue each other over it. So that would be my guess, but no, I have not looked into detail if they're, if they're still taking that open approach.

Yeah. Interesting. I mean, that, yeah, these things can sometimes have very weird feedback effects where, you know, I remember my dad used to say that the thing about a patent is you have to tell everybody how you did it. So even if you're protecting that, you're also giving a lot of hints and people can learn a lot from What you've done, you can have stuff you're not doing.

If you file a lot of patents, you can patent a lot of projects that didn't work out.

Yeah. Interesting. What are the roles of maps in these systems? Does that differ across companies? Like how exact do the maps have to be? How often do they have to be updated?

So this is something that I think Tesla people talk about a lot, and I think is the difference is probably a little smaller than people say. So classically, Waymo had a very detailed map of what's called an HD map that shows at a very fine-grained level, 3D map of all the fire hydrants and trees and whatever in an area, whereas Tesla did not. And there's been a lot of kind of back and forth about this where the claim is that Tesla is better because it can do this without it, but it's not clear it actually is doing it better. But I think that— so there's a couple of things to say about this. One of the, I think, criticisms of Waymo's approach is it's very expensive to collect the map data. And I think that is going to prove overblown because And you can automate it. So the first time you enter a city, obviously you have to have a car drive around and collect the data. But then once you have a fleet driving around, you can just use the data that you're collecting. In fact, I know they did that. Actually, I visited Waymo's facility back in March, and they showed me this hard drive in the trunk that's collecting this data, and they upload to the mothership every night. And so then the data collection becomes trivial because the cars are driving on the roads anyway, and you can just upload the data as it goes by. And then I think you'll be able to— the other step is the cleaning and labelling of the data and integrating it the map. I expect that to get more automated over time. And I don't think this is going to be a big kind of impediment or a big cost factor for Waymo in the long run. And Tesla actually has— they had a tweet a couple of years ago where they— I forget how they described it, but they talked about how they could use their map data to build a map. Now, they weren't saying they were using that for driving, but they're also doing the same thing where they— the obvious thing where as their cars drive around, they collect data and then they build that into a a 3D model of the world that they could use for simulation and for various kinds of testing and stuff. And so I think you'll see some convergence here where— and the other thing is Tesla just obviously uses a map for if you give it an address, it needs to know which roads to take. And so it's following a map too. And I think you can see if you look at the visualization, it has some idea of where the lanes are, even in places that aren't in the view of its cameras. It's less detailed, but it still has a map that it's relying on. I think this is something where the approaches will converge over time.

This kind of goes back to the how far off-road can you go or whatever. Does that mean the cars have to be online to navigate? Like, they can't download the whole map of the whole United States. So do you have a sense for how they are loading maps as they move around?

I don't know exactly how that works, but at least right now, like, Waymo doesn't operate in the whole United States. I think they're loading maps.

Yeah, so they could download the whole map. Yeah.

Yeah, I'm sure they have a big hard drive on there. I think they could probably download all of San Francisco. My guess would be in the long run, there'll be different cards. You'll have a 100-mile radius around your current location or something, and every night you'll download for that. So, yeah, I don't think it requires them to be online. I don't know all the details of how they do those updates and stuff, but no, I think they have a local copy of the map.

So everything is happening locally. This is another thing I wanted to make sure I had a good Read on. There's been some talk. We did an episode with the now defunct Ghost, which was experimenting a little bit with online uses of vision language models. They had an investment from OpenAI, and they were looking at, can we use these vision language models for longer time horizon planning, the sort of reading of the road signs and figuring out which lane I should be in or how to, like, where the drive-through is at this restaurant or whatever. These sort of scenes that are like just higher level analysis required. Are any of the live players doing anything like that to your knowledge, where they actually do make a sort of remote call for any part of their operation?

Yeah, I'm not sure exactly. They definitely do have a network connection and you can do things like call customer service. I'm pretty sure all the safety critical stuff, is done locally. One of the things we haven't talked about yet is the kind of remote monitoring approach because the Waymo cars are quite good, but they're not perfect. And so sometimes they do get confused. And Waymo has said they never have people remotely actually driving the vehicles. There's nobody with a steering wheel and pedals. But what the vehicle will do is they will ask questions or ask for confirmation. They'll say, I'm planning to do X, is that safe? And the person will respond. So I think if a vehicle lost connectivity, it's possible After a certain amount of time, it would slow down and come to a stop because it wants to be able to ask for help. I'm not totally sure how that works. I think there were some incidents with Cruise where that happened, where they had connectivity problems. And so you ended up with the cars with their flashers on getting in the way. And it wasn't that there was anything locally, it was just that the network was down and it was stopped out of an abundance of caution. So there is definitely some interaction with the outside world, but I think they try pretty hard to have it be in a safe state where if you lose the network connectivity, it's not dangerous. It's just at worst, I think they have to slow down and inconvenience the driver.

You mentioned Cruise there. Are they ever coming back? It seems, as far as I could tell, not at the time that they had their very unfortunate incident, which I think is also directly related to this sort of remote help. Right. That my sense at that time was that they were on the same level as Waymo, where they were both operating and Like driverless and had some ability to get help. I wonder if you think that is, was a misperception and they were actually way behind. And, you know, it's crazy to me that they had one safety incident and shut everything down. Like, it seems like it couldn't have been that much of an entirely, like a house of cards. They were, they did have a lot of safe trips too. So what's up with Cruise?

A few things to say about that. So they were pursuing a similar business model to Waymo and were growing at a similar pace. I would say industry insiders all through 2023 were like a little nervous about Cruise relative to Waymo. Like they seemed to have more incidents. They weren't like wildly worse than Waymo, but I think it seemed pretty clear to me they were, their technology was behind, which I think makes sense because they're a few years younger and just, I think they were under a lot of financial pressure because GM has deep pockets, but not as deep as Google's. And a lot of Ford had recently shut down Argo, which was their version of Cruise. A year or two earlier than this. And so I think Kyle Vogt, their CEO, felt a lot of pressure to show that this could be a commercially viable thing. And so he was pushing pretty hard to commercialize and turned out to be— turned out he pushed a little too hard. I would not say it was just one safety incident. The incident that led to their shutdown was a case where a human-driven car hit a woman. Her body then bounced in front of the Cruise car and the Cruise car did not stop in time. And so her body ended up under the Cruise car. And then the Cruise car, after coming to a stop, pulled over and dragged her several more feet under the vehicle. And then when this became a news story and regulators asked about it, the claim of the regulators is that they showed the portion of the vehicle where the Cruise hits the car, but did not show the part with the dragging. Now, they deny that. They say you guys just weren't paying attention, blah, blah, blah. But I'm inclined to believe that. So there was a previous incident where Cruise was accused of Blocking an ambulance where a car got stuck and the ambulance wasn't able to move. And I talked to them. They showed me a video of the incident from their kind of internal God's eye view model. And they said, okay, we'll show you this video, but you cannot take a vid— you cannot take like a screen capture of it. And I was like, oh, can you like send me a diagram so I can sort of read it? Absolutely not. So they were like, I think they were like in defense attorney mode as opposed to like transparency, make sure the public understands what's happening mode. And I think that attitude really irritated. I think understandably irritated the regulators. And so I think it was less that they had one mistake and more that there was a pattern of not being as transparent and clear with the public or with officials or with journalists as they could be. And so they shut down their service. They are trying to come back. I believe they have vehicles with safety drivers operating in Houston, Dallas, and Phoenix right now. Okay. Oh, I didn't even hear that. Yeah, they are planning to come back, but That puts them 3 or 4 years behind. Waymo started doing driverless in 2020, and so they're now several years behind Waymo. And so I think they're probably plausibly still the number 2 company, but they're pretty far behind at this point.

Is there any way for us to get trajectory? I mean, when you say something is like a couple years behind, you've said that about both Tesla relative to Waymo and also Cruise relative to Waymo. It seems like I could make a case. I wonder if you would find it compelling that like maybe they're not really years behind Cruise. Like they have a PR issue. They need to put some safety drivers in. But does that really mean they're like years behind? And for Tesla, maybe I would just make it— you go back to that data case. If they can collect 100x more data or maybe even 1,000x more data, they got millions of cars on the road compared to— I don't know how many Waymos there are, but it's millions, right?

That's definitely possible. Yeah, I'm not saying they're going to— I'm just saying 2020 was when Waymo had an honest-to-goodness driverless service on the road. And it's taken them 3 years, 4 years since then. To scale that up to still a pretty small scale. I'm sure it's possible to do it faster, but I guess I see that as one of the clear milestones is the point where you take the safety driver out of the car is a point where you show a level of confidence in your technology. And right now, Waymo is the only company in the Western world that had that level of confidence in its technology. And so yeah, maybe Tesla will be able to scale it much more quickly once they reach that point, but they haven't reached that point and Waymo reached that point 4 years ago.

Let's run through a couple other live players. I always like to do the live player rundown. I'm very interested in what's going on in China always. And if we could inspire anything in this conversation, I would love to see us race China for self-driving cars as opposed to for weaponized AI or whatever. So what's going on there? Do they have a company that we should be inspired to not be left behind by?

So China has several. I think Baidu is one of them. I honestly have not read much about those because it's just so hard, both with the language barrier and the fact that I can't go to China and the fact that they don't have a free press. And so there's been some rumours that they suppress stories of crashes. Probably China, I think, is the number 2 country on this stuff after the US, and they have pretty substantial self-driving car deployments. But it's not clear to me how much of those are driverless or what the safety case— safety records. Yeah, I can't tell you more than that. And I don't know of anybody who's doing really good in the West who's doing really good reporting on this because I think it's hard to— To tell what the apples-to-apples comparisons are.

At just a notional level, though, do I understand correctly that it is possible to get, like, a Waymo-like experience in China where you summon a car and get into it and there's nobody else in it?

I think there is, yes. And I think they may be at a larger scale in some places than the US. So, yes, I think they are. They're in the same ballpark as Western companies.

You want to do a little rundown just of other notable players in the space? Amazon has a hand in the game, so on and so forth.

Yeah. So there's 2 other companies that are in the robotaxi game with the kind of business model and scale that Waymo and Cruise are. One is Zoox, which was a startup that Amazon acquired a couple of years ago. The other is Motional, which was created— there's a long history, but it was like an Aptiv subsidiary. And then they brought in Hyundai as an investor. And then I think Aptiv got itself bought out anyway. So now I think it's a Hyundai subsidiary. And they are both at the stage of kind of testing the technology, but have not started doing driverless or commercial. And I'm like interested in what they're doing, but I have not written about them much because at this point, I guess, like, I start taking companies seriously when they start having driverless because I think that's now the cutting edge. And so I think they're a little behind, but certainly Zoox, just with Amazon behind them, like they're going to have resources as long as Amazon wants to have them. And so I could see them being a fast follower of Waymo's success. Like people are going to want alternatives. And so that's who they might look to. There's a startup called Wave, W-A-Y-V-E, in the UK that raised, I think, $1 billion a few months ago. And they have an interesting approach in that they're doing the kind of end-to-end approach we've talked about. I believe they actually have a foundation model type thing that has, in addition to driving data, also has some text and images and stuff. And you can talk to your self-driving car. Apparently demos, people have been impressed by demos, but they're again early enough that it's hard to evaluate. I haven't been to the UK and to try it out. But that's, they're certainly somebody worth watching. And their business model is they, I think, are hoping to license to OEMs as an alternative to Tesla. If you've got, if you're Ford or Toyota or somebody and you say, we don't, we don't have our own homegrown version of Autopilot, maybe that Wave will be able to supply them with a competitive version. And then the 2 other companies that I would mention, there's a startup called Aurora that actually used to have the same business model that Waymo's talking, that Wave is talking about now, where they license to OEMs. But a couple years ago, they pivoted to trucking. And there's a couple other companies that are doing trucking, but I think Aurora is probably the most important. And so Aurora says they're preparing to do long-haul, like 18-wheeler trucking routes, I think between maybe Houston and Dallas in Texas. And they say they're going to do it by the end of the year. That's another case where I think I would be nervous if I were the CEO of a trucking company because the failure mode there sounds spectacular.

It's a lot of mass.

Yeah. And so that's not where I would like to stake, but I hope they managed to make it work. And then the final one is a company called Neuro, which was started by a couple of guys who left the Google self-driving car product like 7 or 8 years ago. And they are making street-legal robots. So there's sidewalk robot companies that are very small and slow, but these are smaller than a compact car, like washing machine-sized vehicles that go 20 or 30 miles an hour, and I think could go faster, that don't have a driver or anything. It's just a delivery robot, and it's used for delivering pizza or groceries, things like that. And they've been on the verge of commercialization for several years. It seems like they've done a bunch of pilot projects. And again, it's not totally clear. Actually, they're one of the ones I'm most confused about because it seems like that should be an easier problem because you don't have to worry about safety as much, at least for people inside the vehicle, since there's nobody in the vehicle. I've heard that one of the issues they have is finding, for regulatory reasons, there's limits to how fast they can go because they're in a category that's not designed for freeway speeds. And so finding a neighborhood that has the right mix of a grocery store and then some residential streets, and you don't have to go through any high-speed stuff. I've been told that's one of the issues they're having, but that's another company that's working on self-driving but not building a taxi service.

Yeah, I think that's a great little glimpse of the future. It's just that the form factor of the car could get a little— or of the vehicle could— it's no longer a car at that point— could start to look really different if you don't have to have a driver sitting there in the normal posture. You can imagine people sleeping in cars or whatever, but then also you can just imagine small things that just don't take people at all. I think that possible Cambrian explosion of very different looking things on the road is like a cool future to imagine.

Yeah, 100%.

So let's talk about these safety statistics. Like, where are we on this? It sounds like we probably only are far enough along on a couple of companies to even have a sense. But my kind of qualitative sense has been that the companies seem to be reporting that their products— and here I mean Tesla and Waymo— seem to be reporting that their products are safer than human drivers, seemingly substantially. And then my mental model of this has always been like, I'm not sure I should take that fully at face value, so I'll discount it. But even if I discount it, I seem to I feel like I get back to basically similarly safe to human drivers if I just sort of discount whatever percent. And then if I ask myself, okay, if they're already similarly safe to human drivers, why don't we adopt them more? I assume that it's just a quirk of society that we are going to insist that the AI-powered technologies are not just on par, but are actually a lot safer, maybe order of magnitude safer than human, before we would actually all agree that this is something we should go forward with. How would you complicate that story? Is that a good summary? Maybe you can get more specific in terms of the metrics or how we should be thinking about these. Obviously, metrics are always complicated when you dig into them.

This is super important. Yeah, a lot there. Some things I disagree with. So to start with, let's just set the baseline for what human beings do. So the most important statistic, I think, is that human-driven cars get in a fatal accident about Once every 100 million miles driven. And Waymo has driven about 20 million miles. And so they are not yet at the point that they've had zero fatal crashes, but that is not yet at the point where you can say anything about are they safer, because you would expect a fraction of one death for the number of miles they've driven driverlessly. The other 2, I think, statistics that are helpful, one is crashes with injuries, which are in the ballpark of a couple million miles per crash like that. And police reportable injuries or crashes, which are serious crashes, but not necessarily crashes that have— ones that involve property damage, but not necessarily injuries. And like I said, Waymo has zero deaths, so they have a perfect record, but there's not enough data to really say much. On injuries, Waymo had a report last December where over 7 million miles, they had, I think, 2 or 3 injuries. So that's an injury every 2.4 million miles. I think that works out to. And they did some statistical analysis of human drivers on comparable roads in the same metro areas, same kind of roads. And they worked out to around once every 350,000 miles. And so they're like maybe 5 times safer on a per injury, for injury crashes. And then police-reported crashes, it's 2.5 times safer. It's 500,000 miles per crash for the Waymos and 200,000 for the, for human drivers, again, in very broad terms. And I think that's good. I think that's pretty good. pretty impressive. But I think that's decent evidence that they're safer. But because we haven't gotten to the fatal ones, and that's the ones you really care about, you'd hope that they're all correlated. You'd hope that the fact that they're safer on the lower severity crashes means they're also safer on the higher severity ones. But the higher severity ones are so rare that you can't say that for sure, because there might be some kind of random bug that happens every 50 million miles that gets somebody killed that just isn't correlated with the low. So I think that to some extent, the jury's still out. My guess is they're safer. Probably by a significant margin, but we're going to need 200, 300, 400 million miles of on-road driving before I think you can really say there's a statistically significant—

Yes.

One of the things that— so I've been pretty impressed with the transparency of Waymo. So I mentioned the 7 million mile study. They also did a study when they got into 4 million miles where they partnered with Swiss Re, which is an insurance reinsurance company. They provide insurance to other insurance companies. And so because of that, they have a database of every insurance report ever filed in the auto industry in the United States going back some number of years. And so they're able to do, they're able to look in their database and say, for the specific geographical areas where Waymo's operating, they have a list of like all the crashes and they're able to do apples-to-apples comparisons. And they had a similar thing, 2 or 3 times safer than a human driver based on injury crashes and kind of police reportable crashes. Yeah, that's Waymo.

That's compelling coming out of Swiss Re, who presumably has to ultimately Price the insurance for this, right? Strong incentives to be right in their business.

Yeah. And strong incentive, I think, not to put their name on it unless they think that the evidence is credible. So I think that was a smart thing for Waymo to do is obviously a company puts out data saying we're very safe. You're going to wonder if they cooked the data. And so they can find third parties who are willing to look at the data themselves, maybe supply some of the data themselves, and then put their name on it. That's compelling. Tesla is a hard— I do not think Tesla has been as transparent about this. They have a safety website. Where they say that drivers with Autopilot crash once every 7.5 million miles, which is significantly better than human drivers. The problem with that is we don't know the distribution of those miles. Both, well, for one thing, one of the big things that actually we should talk about a little bit more in general is there's a big difference between freeways and like city streets. Freeways are easier in most ways, like crashes are much less common on freeways. But freeways are also where most of the worst crashes happen, right? The fatal crashes. You're much— if you get in a crash on a freeway, it's much more likely to be a fatal crash. And so the fact that Tesla is much safer, has a lower rate of crashes, might just be that people are turning Autopilot on in the kind of the safest environments and turning it off in like more difficult environments. And Tesla has not done the kind of thing Waymo has done where they've given the data to independent experts or brought in third parties to allow people to try to control for that kind of thing. The other source of data we have about Tesla is disengagements. And so there's a crowdsourced site that where people like upload videos or records of their drives. And right now that's showing that the latest version of FSD has a disengagement about once every 300 miles, which it's hard to know how to think about that because on the one hand, that's pretty good. That's a lot of miles. If you drove a car for 100 miles and it didn't make a mistake, that would, you'd be pretty impressed. But as I said before, Waymo 9, 8 years ago was saying that they were going 5,000 miles between disengagements. So the— The human drivers are actually quite good. And so making a mistake every 300 miles is like not, I think, still not close to human level performance. And I think this is actually, this is a pretty common thing is I frequently see people say, wow, I took a test drive and it was so amazing. And it's how many miles? Well, I did 20 miles like that, just 20 miles a day. If it does a perfect drive for 20 miles, that tells you nothing about how good it is. You need to drive it for thousands of miles to know if it's human level, because the average human driver, 99% of the time, if you do a 20-mile drive with them, it's not going to be. have any mistakes. I think Tesla is definitely making progress. And the other thing is that's crowdsourced. So again, there could be big self-selection issues where maybe people are more likely to turn on Autopilot or FSD on roads they know work well and, or on kind of easier roads. I think it's hard to put too much stock into that number.

It's wild to me that there's not more there. It's just such a strange— it's also like maybe an argument for A lot of the complaints that we hear about everything is overregulated and you can't do anything and we're all just handcuffed by the government. Maybe that's a lie and this sort of puts the lie to that. They're out there with this, without even a good safety report, and we're all just tolerating it.

Yeah. So I should answer the second half of your previous question. Yeah. So one thing that has happened recently is NHTSA, the federal regulator on this stuff, has started requiring companies to report more crashes. And so we now have better data, I think, across the whole country. We used to have only in California. California had good data because the state regulator required it, but now the whole country does. What we don't have actually is the denominator. We don't know how many miles Teslas or anybody else is driving. And so you can probably look and see how many Tesla crashes there have been, but you don't know how many of those, how many total miles Teslas are driven or how many FSD was turned on or where they were driven. And so I would, yeah, I would absolutely like to have more regulation. But I do think, so you were saying, like, it seems like society is, like, resisting this. I really do not think there's much in the way of policy barriers here. California regulators recently gave Waymo approval to operate all the way down the San Francisco Peninsula, down into Silicon Valley. That was several months ago, and they just haven't started doing it. And I think the reason is that they, and this is something we should talk about, is they don't do freeways yet. And I don't think there's any regulatory reason. I think it's just that Waymo is not confident they can do freeways safely. And I think it's a combination of kind of caution on safety, like they're doing the things they think they can do safely, but they don't quite think they have it 100%. But the other thing is that just because they have a vertically integrated business model where they're owning the cars, they're building depots to charge and clean and repair them, et cetera, it just takes a lot of time to just do the work. And so they were in only in Phoenix 2 years ago, then they did San Francisco last year, 18 months ago, and now they're doing Los Angeles and Austin. And they've grown like 3 to 5x in the last year. And if they continue growing at that rate, then in 5 to 10 years, they'll be nationwide. But I think they just, you look back at Uber and Lyft, it took them a few years to go from just in San Francisco to nationwide. And so I think Waymo is growing close to as fast as you would expect for a company that kind of thought they had a successful product. And it's just going to take, because of the business model they have, it's going to take them a while. Now, obviously, if Tesla could just push out a software update and said, okay, everybody's driverless, they can do that very quickly. But I don't think they're ready for that technically. I'm not actually sure if there'd be a legal restriction on doing that. I don't think there would be. I think it's, at least I don't think that's the main issue.

Yeah, that's fascinating because we do hear that so much from Silicon Valley in general, right? This sort of narrative is so often repeated at this point, right? Where everything's, we can't do anything with atoms, right? It's, we all move to bits because we can't do anything with atoms. And now here we have A very regulatable, very suable, at times upwards of a trillion-dollar company that is just doing it and nobody's really telling them no.

Yeah.

And again, even without what I would consider to be just like the sort of safety transparency that I would expect from pure market discipline, honestly, just like a consumer reports basis, I would expect to see a little bit better numbers than it sounds like. Yeah.

I'm surprised about this. If you'd asked me 10 years ago, my expectation would be, okay, they'll build the technology and then there'll be some kind of regulatory process. I didn't know as much about the regulatory state at that point. I think there's a couple of things going on. One is that just the auto industry has traditionally worked on self-certification. So NHTSA publishes a big thing called the Federal Vehicle Motor Safety Standards that just says, here's all the things your cars have to do. And when a carmaker makes a new car, They're supposed to go through the list, test to make sure it meets all that, and then file a report with NHTSA saying we meet all these requirements. And so if you— so what Waymo does is they buy a vehicle that's already certified and then add some sensors and compute in the trunk. But because it's already certified, like, they just meet the requirements by default. And then the second half of regulation is states. So the federal government regulates vehicle design. States regulate drivers, which until recently was human drivers. And driver's licenses and traffic laws and stuff like that. And so there is some variation in the strictness of different states. And so California is one of the strictest. They do have reporting requirements. They do require a license to do various kinds of testing, but it hasn't been that hard to get the licenses. And like I said, they're the strictest. So like Texas is like much more wide open. It's like pretty much anybody can do anything. And actually you do see like Cruise is not going to Texas because I think they found California to be a little too restrictive. But they're not like launching a huge network. They're like— I think they're mostly still worried about, we'll get sued if we get in a crash, rather than regulators come down on us.

And in the Cruise case, did the regulator even tell— my understanding was that they voluntarily stopped their operations without being forced to do so. Is that right?

I think the California regulator did ask them to stop. There was a crash in August involving a fire truck. I think that Cruise was not at fault, but there was some question about whether it could have handled it better, and they were asked to reduced their fleet by half in August as they— and then there was the October crash. California, I think, asked them to stop, but then they voluntarily suspended their other operations where they had, I think, Austin and Phoenix, and they voluntarily suspended that. So it's a mix. There is some pressure from regulators, but it's definitely not like Cruise is anxious to get going and the regulators won't let them.

Yeah. So going back to the original, like, the gold standard of safety data right now comes from Waymo. And is it fair to say that you apply a discount to that? Do you take it at face value? Would you say their reported number is like your point estimate for how safe they actually are?

So it's funny, like the tricky thing actually is that like a crash is a pretty clear-cut thing and they have a lot of sensors and a lot of like personnel. And so I think it'd be pretty hard for them to fudge their like crash data. The hard thing is actually knowing how often humans crash because if you look at the data, They'll have things like, we sideswiped a shopping cart, or we got in a pothole and got a flat tire, or things like that. Or I think there was an incident where like a skateboard ran into a Cruise car that was stopped. Or actually there's a bunch of ones where they're like parked and another car backs up. So all those things are in the Waymo and Cruise safety reports. So we know every time that a Waymo or Cruise car collided with another vehicle or another person. What we don't know is how often that happens to people, because if a person sideswipes a shopping cart, they're not gonna stop and file a police report. They're just gonna move on. And so part of the trick actually has been like collecting the data. So one thing Cruise did a while back is they had this service called Maven that was like a rideshare service or a Zipcar-style car rental service. And they had some of those vehicles outfitted with sensors so they could collect— they would rent vehicles to Uber drivers and then use sensors to collect data. And so they had a database of all the crashes that happened for Uber drivers anyway. So yeah, I'm pretty high confidence in the quality of their data. of their crash data. And I think their safety record is not that much better than the national record, but because a lot of their stuff is in San Francisco, they try to get San Francisco-specific data, which is, I think it's much more likely to have crashes in San Francisco because it's like high, low-speed, dense urban situation. And yeah, I'm pretty— I think that it's very likely true that they're 2 or 3 times less likely to have police-reportable and injury crashes than human drivers. And I think based on that, it seems pretty likely that they are less likely to have fatal crashes, but we can't say much about that because we don't have enough data. Data yet.

So the highways thing remains like one big barrier to super broad adoption, and I guess maybe just even manufacturing bottlenecks and whatever.

Yeah.

But if that is all true and seems very reasonable to at least have a not super wide confidence interval around that, given everything you just said, then why isn't there more enthusiasm for this? If Kennedy were president, he'd be like, giving a big speech about how we've achieved this modern technical marvel and life is going to change. And I don't know if you have a stat on this, but people spend a lot of time driving, right? The pure hours that you could get back in some sense to read a book or whatever, if we could really bring this thing online. We just talked about it doesn't feel like it's the regulators. It feels like it's the culture. What's going on with the culture?

As a person who's written about this beat and wanted people to be excited about this, this is something I wondered about for a long time. I think the expansion, I don't think that the slow expansion is not about lack of demand. I think it's about the stuff we talked about, that basically the technology doesn't quite work and it just, it takes a lot of work to expand it. I think that's the main thing. I think the fact that it's geographically limited does make it hard for people to think about clearly. Everybody's assumption, including my assumption, when they first hear about self-driving cars, oh, it's a car you'll buy and you'll push a button and it'll drive itself wherever you want to go. And for various reasons, that's not how it works. I don't think it's how it's going to work. But then people put it in the bucket of, oh, that's like a research prototype or a thing that's not ready for prime time. So that's one of them. But I also think that the freeway thing is really important. So I think we should talk about this a little bit. So Waymo's safety strategy is basically, if you're not sure what to do, stop. And on city streets, like, that works very well. And then they try to get that rate of stopping low enough that it's not annoying other drivers. And that's worked very well for them. That doesn't really work on the freeways, right? Because if you're going 70 miles an hour and there's a lot of other cars on the road, if you slam on the brake, number one, it takes longer to stop. So you might not be able to stop before you get to whatever the problem is, and then somebody might rear-end you and it causes a traffic jam and stuff. And so that's why it's taken them longer to get to freeways, because they still need to get to what's called a minimum risk condition. But that means getting to a shoulder or getting to an exit. And so that might mean you have to drive for a while. half mile or something. And so that's just a harder problem. And so they're working on that. They started testing on freeways in the Phoenix area earlier this year. And I hope that means that in a few months they'll be able to start, but not commercially. I hope that means in a few months they'll be able to start doing commercial freeways, but they haven't yet. And this is actually another reason they haven't expanded is that it's not a very compelling product if you don't do freeways. When I was in San Francisco, something I would do is anytime I would book a Waymo ride, I would also pull up my Lyft app. And pull up the same ride and look at the time and the cost. It was always a little more expensive and it was always 5 or 10 minutes slower.

And— Waymo is more expensive?

Waymo is a little more expensive. Yep. And Waymo is also slower. If it was along a route where the Lyft would've taken a freeway, it would be a lot slower. Even if it wasn't a freeway, it would still be a little slower. And I think that's just 'cause it's a cautious driver. It'll like follow the speed limit and not run red lights and stuff like that. But it can be a huge difference. So there's a, a video a few months ago where somebody did a test drive of Waymo versus Tesla. In the Phoenix area, they took the longest route you can do in Waymo's service area, and it was like a 2x difference. It was like 20 minutes and 40 minutes because the Tesla got on the freeway and went all the way on the freeway, and the Waymo was going on service streets the whole way. And so it's just not a very compelling product. I think they have plenty of demand because it's a novelty and people— and I think there's, there's some safety benefits and some people like not having the driver, but there's this offsetting factor of it's just not as fast as, as a human-driven car. I think that'll, go away once they figure out freeways. But right now it's like they want to be ready for expansion and they want to be collecting data, and so they are expanding. But I think the really fast expansion will happen after they figure out freeways and some of these other things so that it actually has the chance to be a profitable service. I don't think they're— it's not going to be profitable until they figure freeways out.

Yeah, it seems like that. It's funny, these sort of mirror image approaches of each other. It seems like that can't be too far off. Coming back to the culture or the politics of it, what would a warp speed look like for this? If I was to make a late push for the Democratic nomination, would I show up at the open convention and say, hey, here's my platform, here's what we're going to do to actually bring self-driving cars online? Now, we can suspend disbelief and assume people actually want that. Is there something that we could do as a more society-wide level to say, let's enable this technology? Let's not make it learn or deal with every quirk that we have. Maybe we could clean up our system a little bit and make it a little bit more friendly. What would that look like, if anything?

I don't think there's a ton that you could do. Certainly you could preempt some state and local regulations. I don't think those are a huge factor. But for example, in California, there's been a big fight over individual localities want the ability to license and approve these things, and the state has preempted them. But I don't know if that fight will continue. So having— you can certainly have federal regulation that says you have to allow it In every location and maybe move regulation to the federal level so it's a little easier. I don't think it's a big factor though. Yeah, I've heard you mention on previous podcasts that maybe we should change the built environment. I don't actually think that is very important because the hard thing is not like figuring out where the lanes are, where the stop signs are. These vehicles are all very good at that. The hard thing is that, like I said, the edge cases, the crash sites, that there was a funny case where Cruise like drove into wet cement. I really liked another callback. You had Martin Casado on the talk about like fat tail distribution. It's really that. It's like, There's lots of just weird situations in the world and you have to be ready for all of them. And having self-driving only lanes or putting sensors on the roads or anything like that, like, it wouldn't hurt, but I think it, I don't have any confidence in the government's ability to do that quickly. And I don't think it would add very much value because the, the kind of common cases are not the problem. Like the vehicles already can, if the road's empty or traffic's flowing smoothly, it's pretty much perfect. It's when weird stuff happens. And I don't think there's anything you do to the infrastructure to make weird stuff not happen or make it easier to do. Actually, one, one, one other thing that I do think, so another set of things that have been happening in San Francisco that would be good to replicate is the interaction with law enforcement and first responders is very important. So one thing that Waymo recently added is the ability to, for the, like the San Francisco 911 or the fire department to like geofence an area, say this is a fire zone, so keep vehicles away from it. And then Waymo cars will automatically route around that. And they've also been struggling with the interface. So if a car gets stuck, one thing that's nice is a firefighter can just jump in the driver's seat and drive the vehicle. So having those kind of standards of here's how government can signal cars where to go, and here's how they can take control if it gets stuck, and here's supporting that, that kind of thing would be the most useful thing at the federal level is both do some at the federal level and also develop standards for state and local governments. So I would say that's the big thing, but I don't think this is the main issue. I think the main issue is just like technology and then figuring out the business aspects of it.

It's a bummer. I was, I was hoping that there would be some platform for me to run on.

But it's coming. This is the thing is it's not like nothing's happening, right? Like Waymo, as I said, is, I think they were doing about 10,000 trips a year, a week and a year ago, and now it's like 50,000. So another 5x growth a year for 2, 3 years, and it's going to be a significant thing. Like I would like it to go a little bit faster, but it's not, yeah, it's not the thing like it's stuck. It's just that the US is a big country. So it takes a while to get to everybody.

I do still think of that stop sign. And there's also a true stop sign that's behind a branch of a tree on my way to the highway, which My friend who lent me the Tesla specifically called out and was like, because you live here, that there's a stop sign at every corner in our neighborhood, but it misses that one every time. And so you're going to have to watch out for that. I do feel like there is something there, especially if we're getting into like really long tail things, just make sure the signs are all visible.

There aren't— But, but like Waymo has a map. This is why you need a map. This is why it's helpful to have a map is like. Waymo has the map and I assume that stop sign is correct there. The thing to do there would not be to put, to do anything with stop signs. It would be like publish an accurate database that says, here's all the maps with their GPS coordinates and their whatever. So that could be a good thing is if the government provided data to, to companies that said, here's all the traffic rules and places you're supposed to stop and stuff, that could work. So yeah, I'm not opposed to that. I think it would definitely be like, in general, it'd just be nice if stop signs were not covered up. But that's like a 2% faster kind of thing. That's not the main thing holding things back.

Yeah, gotcha.

Okay.

In terms of the market and how things develop, going back to really where we started, there's these competing visions of a car that you buy and own like a standard car today that drives itself. Obviously, that's what Tesla is pushing toward. And then we've got the robotaxi service. To what degree do you think those remain separate things? Obviously, we've heard from Tesla that they have plans for robotaxi. I don't know what would limit the car companies who want to sell cars. Like, I think they might be scared by the idea that, geez, if utilization of cars goes up 10x, like, what does that do to demand for cars? So I would imagine if a, whatever, if a Toyota or a Ford wanted to do a deal with Waymo, there would at least be some strong reason for them to think, can we make this something that people can have parked in their driveways most of the time instead of actually on the road all the time?

Yeah.

Do you see, like, a conver— is there a possibility of crossover, or do you see convergence, or are these kind of—

So I used to be one of these people who thought that, like, self-driving is gonna mean that nobody's gonna own a car. And I've chilled out on that a little bit because I think people really like to own cars. You've got car seats, or you like to have your golf clubs in the trunk, or you just like the security of knowing, like, your car is in your driveway and it'll be ready whenever you need it. So I do not think car ownership is going away, although I think it will probably get— as taxis get cheaper, which I think self-driving will make it, like, the equilibrium will be such that there's Fewer owned cars and more taxi rides. But the way I envision this market working is I think that you are going to need support infrastructure, maps, but also other kinds of things anywhere where a level 4 self-driving car operates. And so I think what you're going to see happen is Waymo will expand nationwide. You'll have a taxi in every area. And then once Waymo's service area is 50%, 60%, 70% of the country, then Waymo can go to an OEM and say, how about if you add— we'll add Waymo service. Where it's $100 a month and then your vehicle has built in like the Waymo service where you can subscribe to that and then you push a button and it'll drive itself. And I think that'll probably take 10 years to get to the point where Waymo's able to offer that. And maybe somebody like Wave will get there independently. Going the other way, I'm pretty skeptical of. Tesla claims they're doing robotaxi. I think you need that infrastructure. Teslas are gonna get stuck sometimes, and if there's nobody in the car, because what Elon has this vision that you'll buy your Waymo and then you'll go on vacation and while you're on vacation, your car will be driving around earning money for you. But what if your car gets a flat tire? Like, you're not going to come home from your vacation and fix a flat tire. Like, presumably Tesla is going to have to send somebody. And it's not like it's impossible for Tesla to build that infrastructure, but that's like not the business they're in. And they certainly, I don't think, have started like building that infrastructure. And so I think it'll take a few years at least for Tesla to be ready to offer a Waymo taxi service. And yeah, I'm not sure they have the corporate culture or the infrastructure or anything to do that. And we'll see. But I guess I'm not that optimistic about the robotaxi part of that. I think that they're more likely to stick with the service they have now, and maybe at some point they'll be ready for driverless, but I think it's probably a few years away still.

Who do you think does have that culture? Because I wouldn't think of Google as being like the obvious company to really want to handle all the flat tires.

Uber. Yeah, no, I think this is a— I think this is going to be—

So you're putting that in the driver's laps, right?

Yeah, no, I think it's probably going to be I think it's a, that's a big open question. Definitely. I do think one of the other reasons that Waymo may be growing relatively slowly is that Uber has, or that Google has always operated in high margin businesses where massive scale software. And so they just pay top dollar for the best talent and don't worry too much about like individual productivity. They're not like scrappy the way Amazon is. And so I could see, totally see a scenario where they, their technology works perfectly. They're not making any like mistakes, but it's just everything costs like 50% more than it would if Uber were doing it. Or you actually see this to some extent with Uber and Lyft as well, where Lyft takes like half of every fare. And it's because they have this building of 5,000 software engineers. That's actually like a large fraction of Lyft's revenue. So I do think that companies are going to have to figure out how to be a little leaner for it to run in this kind of more cost-conscious market. And I can imagine a number of ways, maybe Waymo will have some kind of franchising thing. So a while ago, they had a deal with Avis to help them manage some of their vehicles in Phoenix. I think this was the 2018 time period when they thought they were going to scaled rapidly. So I could see them either licensing their software to other people or having some kind of franchise model or subcontracting. I don't know. There's lots of different ways you could imagine, but I'm sure that there won't be like the nationwide Waymo network. I doubt it'll be like all like Waymo employees doing everything. I'm sure they will have some kind of partnerships with other country companies. Uber is a good example. I do also think getting the two-sided market of rideshare to work and build it up is very expensive. Uber and Lyft spent a decade spending billions of dollars to do that. And so one approach would be just you make Uber the partner, right? And I don't know if Uber owns the vehicles or maybe there's a third company that owns them, but as the ride-hail app, like maybe in the future, like actually most of the Waymo vehicles are dispatched through Uber and Lyft. I think that could be fine. So yeah, in terms of market structure, I think it's very much an open question. And I don't think Waymo, because it's not actually a profitable business yet, I don't think Waymo is like thinking super hard about this. The easiest thing for them to do is just to run their own ride-hailing services and maybe that'll work. Once they get to, okay, like, this is actually like a viable service, which I think is probably a year or 2 away, then they'll start thinking about how do we scale up really fast? And that probably will mean finding partners or thinking about other business models.

Yeah, Amazon definitely comes to mind there as somebody that already manages like a million vehicles and has a pretty good reliability rate for making sure the package ends up on my porch and dealing with it when it doesn't. So in some ways, that's really in their wheelhouse. Yeah.

I could see a world where a year from now Zoox starts doing driverless and then they scale it much faster than Google. And so they end up being as big as Google, as big as Waymo, 5 or 10 years from now. Yeah, that's one reason I wouldn't count them out, is they have a lot of that expertise that Google doesn't have.

Are there any other surprising scenarios that you think are underestimated? Like, one that I just came up with in preparing for this was Tesla currently charges $15,000 for the feature. An argument could be made that they should just give it away to scale even faster and try to dominate and then figure it out later. They can always charge again later. I don't know if you think that is at all realistic. And then I also wonder about just weird partnerships, alliances. This is something I've been really struck by in the context of AI in general, is just how we've got OpenAI partnered with Microsoft and Apple at the same time.

Like, what?

I never thought I'd see a single core technology provider could play both sides of that divide. So, yeah. Is there any— and also, at the same time, their core product is free, right? You can go to ChatGPT and use GPT-4o for free, and yet they've managed to do these strategic partnerships. Is there a move there that Tesla could make something free and partner with Ford or whatever?

I don't know. I don't think that the cost of FSD is their main Bottleneck. The $15,000 is just a way, like when Tesla was in kind of desperate situation, 3 to 5 years from a cash perspective, this was a way to raise cash. And they are shifting to a monthly model. Like you can get $100 a month FSD, and I think that probably makes more sense in the long run. It's $100 now? I think that's right. Yeah. Per month. I think that probably makes more sense. It's gonna be a service, not just like a piece of software you buy one time. So I think that does make sense to have a recurring kind of revenue model. In terms of like other ways the market could unfold, to some extent, I think that the kind of Crazy Wild West, bunch of money flowing in period that in self-driving was that like 2016 to 2018 period. And I think there's been like, I think it's like probably 5 years behind the, the kind of generative AI. And I think that there was some of that kind of stuff happening back then, but then a lot of those projects just went out of business. And I don't expect to see a lot of that the way we do now. Obviously, once, if Waymo becomes very successful or Tesla or somebody.

Yeah.

Then a new generation of startups might come in. But right now it's just, I think, in a consolidation mode where the companies that are left are just hanging on, trying to survive. So the non-Waymo ones, it's just, can they get to market at all? And so nobody's going to want to put new money into them until they show some success. And the question is, yeah, can they survive or do they go out of business?

There's sort of an analogy there to the large language model market, I would say, which is that this has happened in compressed time, but Just over the last couple of years, you've seen this obviously emergence of very notable leaders with compelling technology, a bunch of people who have raised very significant money to try to get into the game. And it seems that we're already hitting a period where that second tier is being thinned out. It's almost like the history of self-driving has rerun for large language models in 10% of the time, or maybe 20% of the time.

Yeah, the current large language model market feels a lot like that early self-driving market to me. There's a lot of, I have trouble working out the kind of math of the 2nd, 3rd, 4th tier LLM companies, like how they're gonna make a profit. Like I'm sure one or two of them, but there were a lot of self-driving companies that were just like, we're doing self-driving. And they were like the 6th or 7th best self-driving company and they didn't make it. And so I would not be surprised. And I also see a similarity in that this kind of long tail problem where people are like, LLMs can do 90% of X, and how hard can it do the last 10%? And what you learned with self-driving is that last 10% or last 1% is really hard. And I expect to see some of that. I don't think we'll see the same degree of crash because clearly there are some things, customer service automation and various kinds of documents. So I think there clearly is, are uses for LLMs. I don't think they're going to go away. But yeah, the crazy valuations I think are going to be hard to justify.

And I definitely expect some consolidation or some of the kind of second-tier companies to go out of business in the next Yeah, I think that there's a musical chairs game that's already underway, and the music doesn't seem like it can play that much longer for a lot of these second-tier language model companies, which is not something I take any pleasure in coming to that conclusion, but it does seem hard to argue otherwise at this point.

Yeah.

I'd be interested to hear a little more. I know you have a map that goes a little deeper Even than what you just said on what your expectation is for future of AGI and whatever, I would put a little pin in that. I think it is a lot easier to control the environment for language models. And that is a skill set that if we do in fact see this stall out that some are predicting, which I don't predict, to be clear, as I think, yeah, we're just stuck at GPT-4 plus a bit, then what I would expect to happen over a long time is this kind of FSD sort of slog, except instead of grinding out for all of the additional nines that you need for a functional FSD product. I think what we would see in the language model side is a lot of systematically sanding down rough edges of different business processes so that you can reliably feed the language model something that it can handle. And in that way, you could probably automate still a ton of work across the economy. It's just that everybody's in this moment now where we're not used to the technology. And there's the expectation that it's going to get a lot better and a lot easier. So do I really want to put in all this elbow grease now to scaffold this thing up and make sure that I have it dialled in when I can maybe just wait for GPT-5 and maybe it'll solve all my problems? But it sounds like you don't think that GPT-5 is going to solve, or even 6, is going to solve all our problems based on your decade following self-driving.

Yeah, I don't have a super strong prior about this. And I do think the one obvious difference between LLMs and cars is LLMs don't kill anybody. And the really hard thing about self-driving is it has to be close to perfect or you don't have a product at all. Whereas with LLMs, there are, as you say, there are some, either some narrow domains where you can get it perfect even now, or domains where there's a human being checking the output. And so it's fine if it makes mistakes sometimes. And so I definitely expect there's enough, like you said, enough business processes and kind of small-scale stuff that there'll be a profitable business for somebody. But the hopes of the really big stuff where we're going to have AI doctors or AI lawyers or programmers, things like that. Like, I guess one of the similarities I see is that the, I think people may be underestimating how much the hard stuff is different from the easy stuff. People see, okay, Tesla can drive on the freeway. And so we're like almost, how much harder can it be to do cities or to do crash sites or things like that? And similarly, like the routine doctor's visit, I think you can do pretty much with kind of a checklist. But then part of what's valuable as a doctor is they'll recognise, oh, this is a really weird situation, and there was information from a checkup you did 2 years ago that is relevant to this. And that long context, like abstract reasoning kind of thing, it seems to me like LLMs are pretty bad at that still. And I'm not going to say that GPT-5 definitely won't do that, but just scaling up the transformer does not seem to me like it's necessarily going to do that. I think there's going to need to be some kind of deeper improvement to how these things work. And I think that might take a few years. I'm very interested in your Mamba piece, or your Mamba episode back in December got me interested in that. And I'm working on a piece about Mamba. I can see that being one piece of that, and that seems to be several years from being at large scale. But anyway, yeah, I have a lot of uncertainty about this, but I guess I've just learned to be sceptical about confident claims that, oh, this, if we just scale this up or keep doing this, it's going to, everything's going to work. Sometimes that's true, but often it's not.

Yeah, I think if there's one thing I've learned, it's— and I recall Professor Mike Levin saying this about a very different subject, but he was like, if there's one thing I could say is that we should not be confident because we've been repeatedly surprised and it seems like we'd be— there's no rational basis to expect that the surprises will stop on— it could go in either direction.

And I think we've had a period where the surprises have mostly been on the upside, as though these large language models work better than we expected. In a similar way, Like in 2014, 2015, 2016, self-driving cars kept pretty quickly. We went from like nothing to like having things that were like pretty close. And but then you can enter a period where the surprises are the other direction is now we've updated our expectations. We were optimistic and then, oh, actually that was right. You're on the kind of other side of the S-curve. So I don't know when we'll hit that, but I'm just like, based on my experience, I'm expecting it for periods like that to happen. Yeah.

Honestly, I think it would probably be In some ways for the best. There was a little bit of a just longer window of time for people to absorb what technology already exists before just getting washed over by yet another wave of it. Maybe last thought, and then I'll give you a chance to, if there's anything we didn't cover that you can, that you want to touch on, we can add, but I'm in Detroit, obviously famous auto town. You're in DC, famous rulemaking town. Which one of us do you think gets robotaxi service first?

Oh, me. No question.

You think they want to go there to impress regulators?

No. Waymo has started testing here. I think that— so several things. One is that cities are easier. We have very few freeways here. I think probably Detroit has more freeways.

Got a lot of freeways.

Yeah. 2, they are testing here. The 3rd one is snow, which I don't think is going to be a big obstacle, but I think If you look at the places Waymo's gone so far, it's like California, Arizona, Texas. So I think they'll go from the south to the north. And so, yeah, I expect all the southern cities to have it before any of the northern cities. And I think DC is probably southern for that purpose because we only have 3 or 4 days of snow.

Breaking my heart. I don't know if this is something I'm quite ready to move for, but an advocate for being willing to change the environment, this is maybe something I should at least consider on some level.

If you can come up with a policy agenda to get rid of all the snow in Detroit, I think that would be pretty popular. But yeah, no doubt difficult other than like massive global warming.

It may imply other downsides. Cool. Well, this has been a great run-through of an industry that I'm super interested in, and I think we're both hoping is really going to start to work sooner rather than later. Anything that we didn't touch on that you want to make sure we mention?

No, I think we covered it. I really appreciate the thoroughness of these episodes. I really I rely on you for situational awareness of a lot of things that happen in the AI industry. So it's been fun to be at the other end of that process. Cool.

Thank you. I appreciate that. And it's a lot to live up to, but I will do my best. For now, Timothy B. Lee, author of the Understanding AI newsletter, which you can find at understandingai.org. Thank you for being part of the Cognitive Revolution.

Thanks, Nathan.

It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at tcr@turpentine.co, or you can DM me on the social media platform of your choice.