EP 234: Driving the Future - NVIDIA's Vision for AI-powered Transportation

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life. What does the future of transportation look like, especially when AI is getting more and more involved? Well, you know, are we going to have flying cars? Is everything going to be self driving? I don't know. I don't have all the answers. But today I have a very special guest who is going to be able to tell us what the future of AI powered transportation looks like. All right, what's going on? Welcome everyone. Thanks for joining us. My name is Jordan Wilson and I'm the host of Everyday AI. We're a daily livestream podcast and free daily newsletter helping everyday people like you and me not just learn what's going on in the world of generative AI, but how we can all leverage it to understand the world around us and to grow our companies and to grow our careers. So if you're joining us on the podcast, thank you. If you're joining us on the live stream, it's great to have you at this special time. Not our normal time, but we're doing double duty here because we are reporting live from Nvidia's GTC conference here in San Jose. I'm extremely excited. I'm looking over the show floor. The future of AI is happening literally right outside this window at Nvidia gtc. And if you feel like you missed out, don't worry, you can still sign up right now and attend the conference for free virtually. If you missed out on some great sessions like the CEO Jensen's keynote, you can catch that as well as so many other great sessions and workshops from world and industry leaders in all different aspects of of artificial intelligence. And if you do sign up, make sure to check out the link in our show notes so you can enter as well to win an Nvidia GeForce GPU as well as DLI learning credits like y' all like there, there's literally the experts, leading experts in the world like our guest today that you can go learn from. So make sure to check that out. But I'm excited now to talk about the future of transportation and AI. So please help me welcome to the show. Let's bring them on. There we go. We have Danny Shapiro, the Vice president of Automotive at Nvidia. Danny, thank you so much for joining the Everyday AI show.

It's great to be here.

All right.

Hey, can you.

Yeah. Can you tell us a little bit about what you do in your role as Vice president of Automotive at Nvidia.

So our team is really focused on revolutionizing the transportation industry. And so we're working with hundreds of car makers, truck makers, robo taxi companies, shuttle companies to make transportation safer and more reliable. We're bringing supercomputing into the vehicles and adding extra safety and convenience features through sensors and supercomputing. And it's just, it's really a lot of fun. What we get to do, we work with a lot of different companies, but the reality is the roads are dangerous and AI is going to make us all safer.

Yeah. And Danny, can you maybe explain this a little bit? Because you know, I'm thinking I'm like, hey, my, you know, 18 year old Honda probably doesn't have any Nvidia parts in it. Right. But, but how, you know, specifically is Nvidia? Because a lot of people still think of Nvidia as a GPU company and it's obviously so much more than that. But how specifically does Nvidia work in the automotive space? You know, is it the, the newer cars self driving, how exactly does that work?

That's a, it's a great question. So we started working with the auto industry more than two decades ago and, and it really is kind of the types of things you're alluding to originally is graphics, right? People would design cars using Nvidia graphics. The engineers would use their, you know, computer aided design tools running on a workstation with Nvidia inside. And we would do all kinds of other, you know, crash tests and things like that in simulation. So that was really the early days. Then we started bringing our graphics inside the car. And so back in the early 2000s, this push of consumer electronics was taking off the iPhone, tablets. But the electronics inside vehicles were pretty antiquated by comparison. So we helped pioneer bringing the touchscreens to vehicles. Digital instrument clusters, really, rear seat entertainment, the head up displays. Again, anywhere there were pixels, you look to Nvidia to bring those into the car. Now one of the things we had to do was really transform ourselves as a company into an automotive grade company. Meaning if you have a mobile phone and you leave it on the dashboard of your car on a hot summer day, it won't operate right. You get that little alert. But inside of a vehicle, whether it's super cold temperatures in the winter, hot summer days, it has to operate. And so there's also a lot of other harsh conditions inside of a vehicle. The shock and vibration and dust. So we basically create a version of our products and in this case it's the Nvidia Drive product that is a system on a chip, we call it soc, that integrates a cpu, a GPU, and a number of other processors on a single die to handle all the different types of computation that you would want to do in a car. It's a little different, of course, than a phone or a laptop or a data center, but that's the flexibility of Nvidia to have this architecture that can scale across many different platforms.

Can you, can you tell us, you know, what is, what is new, what is happening? You know, I'm here at the conference, you know, in San Jose, and I do see, you know, vehicles all over here with, with the Nvidia signs. I mean, what is, what is new and what has just been announced, you know, with Nvidia here at GTC in.

The automotive, again, another great question. So, yeah, we have vehicles all around the show floor. There's a semi truck parked in front of the convention center powered by Nvidia. We have Mercedes Benz in the lobby, their brand new CLA concept car, which is the precursor to their next generation of vehicles that will all have Nvidia processors inside. Lucid is there Volvo Polestar. These are all brand new cars that will start shipping later this year. And so what we have then is essentially this supercomputer, an AI brain inside the vehicle. It's connected to different sensors, cameras, front facing rear side cameras, radar lidar, which is a laser scanner in many cases. And so all of that information is helping the vehicle understand what's around it. It can sense the lane markings, of course, see signs, traffic, street lights, other cars, bicyclists, pedestrians, fixed objects. And it takes all that data, brings that into the Nvidia Drive system, which then has to figure out where it's safe to drive and ultimately control the vehicle. Steering, accelerating and braking. Go ahead.

No, no, no. Yeah, sorry, sorry. Go ahead. Yeah, Talk about. Danny, what you were just saying about announcements.

Yeah, so at the keynote, we announced that byd, which is the world's largest EV maker, is adopting our newest product, Nvidia Drive Thor. So that's our next generation supercomputer that's four times more powerful than the one we have in production right now. Be available next year. And this is going to enable all kinds of new AI capabilities for sensing outside the vehicle as well as inside the vehicle. So knowing who's in the car, are they paying attention or not detecting if there's anything left behind in the car. So there's just a whole new development going on with Respect to software that will be able to control many different aspects of the car.

Yeah. And hey, for, for any of our livestream audience like Douglas here who's interested in automotive. Go ahead, ask a question. You know, we'll, we have one of the leaders in the automotive industry joining us live today. But you know, one thing Danny, that you brought up there is, you know, a lot of these new advancements with Nvidia's chips. So you said the Thor is, you know, giving you about 4x the power. You know, we even saw on, on the kind of consumer side with, with the Blackwell and you know, the, how much more powerful. You know, even this generation of chips, Nvidia was already the leader and then you come out with chips that are, you know, exponentially more powerful. So I mean, what does that even mean for, you know, the future of transportation, Right? Like is there a certain limit, right, where, where your consumer vehicles can't even use the power of these chips? Or does this just mean that we are going to be seeing new capabilities in our cars that maybe we just had no clue were possible?

No, you're absolutely right. I think if you look at the history of computing, whether it's personal computers, supercomputers, mobile devices, there's never enough compute, right. The software gets developed to take on new features, new capabilities, but it always then hits that upper limit and then the next processor comes out. And so that's where we continue to innovate, create higher performance systems, lower energy consuming systems as well, because that's really key, especially in an electric car. But I think the thing to realize here is more computation equals greater safety. So as you have more cameras or more sensors on the car, higher resolution sensors, more complex algorithms, there's just a lot of diversity and redundancy required to ensure safety. So the more compute you have, the safer the vehicle can be. And so, for example, we can have a system that's detecting pedestrians and that's going to take up a certain amount of computation. But then we also have a neural net that's looking for what we call free space, the absence of objects. And so these are two different algorithms, but together they ensure. It's like a double check, right? We make sure there's nothing in front of us and if we see things, we know to avoid it. And that just requires double the computation because we're running two algorithms. Then multiply that by lane marking detection, sign recognition, pedestrian detection. So there's dozens and dozens of of these neural networks that are running in the car to make sure that we're able to drive safely.

You know, Danny, speaking of that, you know, adding, you know, these, this, this new function functionality on the road that maybe wasn't possible before. It seems like the industry has really been working, you know, toward the, the full self driving and autonomous driving for a while. And you know, different companies are finding different levels of success us. Do you think now that maybe, you know, compute is, is maybe less of an issue for, for what is capable? Do you think that over the, you know, the coming months and you know, year or two, are we finally going to see the point now where, you know, full self driving and autonomous vehicles are more of the norm versus the exception?

I think so. I think that it still is going to take some time to ramp up, but the industry really underestimated the complexity here. If you go back 10 years, let's say we're in 2014, 2015. All the automakers were predicting by 2020 we're gonna have full self driving on the road. So we were driving towards that goal. It's hard. It's really, really hard. This is perhaps one of the most complex AI challenges in the world and lives are at stake here. So safety is key, so we wanna make sure that we get it right. And so we're really not pushing for a specific timeline. We're pushing for a level of safety that far exceeds what humans can achieve. And so there's a number of tests going on in different regions with car companies, with trucking companies, with robo taxi companies. And we're seeing great success. And it's a global thing we're showing we rides robobus on the show floor and they're operating autonomously in Beijing, in Shanghai, in Abu Dhabi. So there's different trials going on around the world. Aurora, their truck is driving autonomously in Texas, on the highways in the Bay Area. There's a ton of tests going on from a wide range of companies here. It's all powered by Nvidia. But I think the key thing is to ensure that we can handle the basics. It's those things that almost never happen that will cause the system to trip up. So one of the things that we're doing is using simulation to be able to train as well as test these vehicles. So it kind of goes back to our roots a little bit in video game technology. But we'll create virtual worlds that are digital twins of the environment. So the roads, the buildings, other objects that are fixed, plus the other cars, trucks, pedestrians, bicyclists, motorcycles are all in the simulation. And so we simulate the actual signals that the cameras, the radar, the lidar would be generating the real car. And we run that through the computer that's in the data center, but it's a digital twin of what would be in the real world. And so the computer that's in the data center doesn't know it's not on the road. It thinks it's driving on the road and making driving decisions. And so that way we can create all kinds of hazardous scenarios without putting anyone in harm's way. We can test what happens in different weather conditions, different road conditions, you know, having a virtual child run out in front of the car at night and do we detect that? So it gives us amazing ability to ensure the safety of vehicles and the software before we put it on the road.

Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on gen AI. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use Gen AI. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you can partner with us too. Just like some of the biggest companies in the world do. Go to your everydayai.com partner to get in contact with our team or you can just click on the partner section of our website. We'll help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on gen. Danny, can we talk about that a little more? Because I think at least even for me personally, my eyes are on the Omniverse here in the keynote. Seeing so many great real world applications for safety, climate change, so many different applications that are positive. But can you just talk a little bit about for our audience that maybe isn't super familiar with the Omniverse and also now with increased computer, because now I'm guessing you can run way more simulations simultaneously because you have more computing power. Sue, can you talk a little bit about what you're even capable to do on the safety side with this increased functionality of the Omniverse?

Omniverse is our platform for digital twins and simulation. We can make it. We do make it available basically to every single department in. Let's just take an automaker, for example. The designers can be working in a variety of different tools and bring their designs into Omniverse and be able to collaborate. So somebody can be working on the wheels in one software application. Someone else can be working on the body designer in another. Maybe somebody's working on the headlights. Omniverse lets them bring it all together and collaborate and see what everything looks like. So this is in photorealistic rendering and using real time ray tracing, so you get all the reflections and. And it just, it looks beautiful, the materials. Right. And so that then can be shared with the engineers who are trying to figure out, well, how do you build this thing? And then so they can have a common design and they can then go back and forth and collaborate. We can bring that into all kinds of structural engineering and bring it into a virtual wind tunnel simulation. So a designer can create something and you can see, well, what's the coefficient of drag, how, how fuel efficient will this design be and how easy it is to manufacture. We're working with a number of automakers in BMW, Mercedes Benz, who are creating factories. The entire factory is going into Omniverse now. And so we can model the layout of the factory before the factory is even built. We can see how the robots are all going to interact. We do a simulation of the car being built. And so, yeah, you can see just a still here showing this entire factory. It's a massive amount of data. Every robot here now is following its programmed path and will operate. And so we can make sure that two robots, as they swing, they don't collide. So we do that in simulation first as opposed to then realizing on the factory floor that there's a problem. And then you have to move things around or move a wall or make the ceiling taller or whatever it is. So this is just an amazing tool. And the amazing additional part is that that factory runs in simulation. So we can see the vehicles being built in the actual simulation. And so then in the real world, it makes it very easy to bring the factory up.

Yeah. And you know, if you're listening on the podcast, we just have a photo here that kind of shows the concept of, you know, what this can look like in the Omniverse, kind of being able to replicate what is actually happening in a real world manufact plant and being able to replicate that digitally in the Omniverse. So, you know, I'm wondering, you know, Danny, like what's. And we don't have to go into specifics. But, you know, in general, how much of the decision making behind the future of automotive. Because, you know, obviously, you know, Nvidia is not making cars, but you're working with, you know, some of the world's largest car manufacturers. But how much of the actual development of future safety, future AI technology, how much of that is actually originating from the Omniverse versus maybe kind of real world testing?

It's a really good question. And it's moving in the direction of everything will be simulated first. That's the principles we're using at Nvidia in making our chips. We simulate it all. Our new building, you were just at our headquarters that opened recently. That whole building, it's a smart building. It's kind of a living entity. It's software defined. There's all kinds of systems monitoring. We simulated everything in that building before it was built. The angle of the glass and the position of the sun. It's designed to let maximum light in, but still not create a lot of heat inside the building. And so we're able to model the physics of systems now in Omniverse and use that to build better products.

And hey, a great question here from Douglas. Douglas, thanks for this question. He's asking how does the AI for a vehicle differ for on premise versus a connected vehicle? If I drive a vehicle in the mountains, will the AI stop? I wasn't even thinking about that, but yeah, Danny, how does that work? Right? Like, is most of all the AI being on premises? Yeah, go ahead.

It's a super good question. So I think it's maybe a little bit of a misconception, too. So connected cars, everyone talks connected cars. And I think that's great, right? You want to have a cellular connection because you get your NAV updates. You can check the weather. You can talk to Siri if you want to. You can stream Spotify for autonomous operation, though, and for safety, nothing goes to the cloud. It's all. It has to all be on board. So you can't rely on your pedestrian detection system having to go to the cloud to decide if that's a pedestrian or not. It takes too long and the connection's too unreliable. So all the sensor data goes straight into the supercomputer, our Nvidia drive system on board. And those decisions are made in a fraction of a second. We're looking at video, which is 30 frames in a second. So we have 1/30 of a second to analyze all the pixels in those frames of video and decide what's a pedestrian or What's a car or what's a lane? The connectivity piece then is software updates your cars parked overnight and you could get an update just like your phone, which would add new features and new capabilities. And then if you're trying to use a system, say, hey, I'm interested in finding a good Japanese restaurant, then it would go to the cloud and get that data and come back.

It's, it's something that I, I don't really think about a lot, Danny, but it sounds like, you know, for the most part, especially if you have a newer connected car, you know, powered by Nvidia, that your, your, your car is probably smarter than your, you know, maybe your computer. If you're using a basic laptop or a standard cell phone, it sounds like, you know, whatever is, is the, the AI powering your, your car might be just as powerful or maybe more powerful. Is, is that the case?

Oh, absolutely. Again, it's tuned for specific applications. You know, it has, the software is designed for sensor processing in this case. But our new Drive Thor, it's using our Blackwell architecture. So those really powerful GPUs that are going into the data center, that same architecture is leveraged and goes into Drive Thor. And this will be a 1000 ton hops processors. That's 1000 times a trillion operations per second. So it's a massive amount of compute for artificial intelligence and sensor processing.

Yeah. And one thing that I'm thinking about now, Danny, because I've even heard it from certain people I've been talking out to on this exhibit floor and people are saying, hey, now with Blackwell and all this additional computer, you know, some people are rethinking, you know, what's possible with their product or service. You know, I'm even curious for you, you've obviously known about what's going on at Nvidia, so you've had time to think into process what all this additional compute that's been announced here at GTC actually means. But what do you think that it means outside of what's directly in front of you? People are always saying, oh, are we going to have flying cars? Is every single vehicle going to be, you know, fully self driving? But what does the, the future beyond tomorrow look like with all of these new AI powered announcements?

So obviously the, the big trend and it's kind of taking over all industries is generative AI now the ability to have artificial intelligence. Not just do, you know, pattern recognition, which was kind of essentially the, the past wave being able to identify things, but now take content and create new versions of it. So ChatGPT, you type in text and text comes out. We're going to see a lot of large language models running in the car. So you'll be able to speak naturally to your car and your car will answer intelligently. And that will happen locally on the car. So we'll get trained on everything with your car, the brand, you could ask it any kind of question. It can also maybe tell you about things that it's sensing in the car. Maybe there's a new rattle or something so it can understand what's going on and maybe alert you or go ahead and just book a service appointment. And if it is self driving while you're sleeping, it would drive the car back to the garage and have that repair made and then it'll come back to your house in the morning. We do believe everything ultimately will be automated and autonomous. And it's a process that's going to take a long time. There's lots of, lots of cars in the world and a lot of them don't get refreshed for many, many, many years. But we're working with a lot of automakers to bring this technology to market, bring it to consumers and make it affordable.

So Danny, in your role at Nvidia, you have your hands on really the future of transportation, right? Like Nvidia is working with the largest hardware and other software manufacturers in the transportation. So I'm curious, what are you most personally excited about in the future of transportation?

I think, you know, you look at your phone and you get, you know, periodic software updates and all of a sudden the interface is better, there's, there's some new apps and it's like, it's kind of, you get a sense of joy from getting things getting better. And I think that's really the what's going to happen with your car. It's not the, the old model where it was the best it's ever going to be the day you drive it off the lot. But moving forward, you have a software defined vehicle. You have Nvidia drive inside a very powerful computer and it will just get better and better with each software update. So your next car you may buy and it doesn't have this fully autonomous capability. But during its life, as the software is enhanced, as regulation changes and autonomy becomes the norm, your car can get a software update if it has the sensors and the compute on board to handle it. So maybe first there'll be a highway autopilot, then maybe an urban autopilot, maybe there's valet parking. You can go to the restaurant, go to the movies, you get out of the curb and your car is just going to go park itself.

You know, we've, we've covered just about, you know, so many different things, you know, both from recent advancements that Nvidia has been making, you know, the future of full driving autonomous vehicles. But, you know, Danny, as we wrap it up here, what's, what's the one take home, you know, piece of advice or message that you really want people to know specifically about how Nvidia is powering the future of transportation with AI?

Yeah, well, we call it, you know, an end to end approach. And it really starts from that design of the vehicles. We're helping automakers and truck makers create more innovative designs, more efficient designs and safer designs through the engineering and manufacturing, the testing, the simulation, and even retail. Being able to take all that information through to the consumer and help them make purchasing decisions and, you know, streaming a car configurator in their web browser on their mobile device and even take a virtual test drive. So the buying experience and then even the service and maintenance omniverse is going to play a large role there too. And so we're just really excited by the technology that we're bringing to the market and seeing that now our customers and partners build on it and really create cool products in the future. I think that many people are potentially afraid of autonomous driving. There's a lot of sensationalized stories in the press, but the reality is that there's a lot of hazards and dangers on our roads and a lot of accidents, injuries and fatalities every day. And as a society kind of come to accept it. But I believe this technology is going to dramatically reduce those figures and make the world and our roads much safer.

Danny, you've helped us so much better understand what's happening in the transportation and automotive industry and even walked us through everything new here. All the new announcements with AI at Nvidia gtc. Thank you so much for your time and joining the Everyday AI show. We very much appreciate it.

Pleasure. Great speaking with you guys.

All right. And hey, there was a lot there. Don't worry if you missed it. If you're out, you know, on your walk and maybe your reception cut off for a second. We recap every single article, every single episode on our website. So make sure to go to your everydayai dot com. We're still going to have more information in there about what's happening here at Nvidia gtc. There's so much we couldn't get to, so we're going to have more in each and every newsletter throughout the rest of this week and probably early next week. So if you haven't already, go to your everyday AI dot com, sign up for that free daily newsletter and we'll see you back tomorrow and every day for more Everyday AI.

Thanks, y'. All.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit your everydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.