Nvidia’s CEO On What It Takes To Run An A.I.-Led Company Now

AI is transforming the world, and it starts with the right compute. ARM is the AI compute platform trusted by global leaders. Proudly NASDAQ listed, Built for the future. Visit arm.com discover. Welcome to the hbr ideacast from harvard business review. I'm alison beard. There's no question that the biggest business story of 2023 is artificial intelligence. Not just all of us using generative AI tools like ChatGPT on our own, but what those advances mean for how companies can collect and use data to improve decision making, R and D and more. These developments are driving a new, exponentially faster era of computing, and our guest today is at the forefront of this warp speed transition. Jensen Huang is the founder and CEO of Nvidia, a world leader in AI driven computing that makes both software and hardware for a host of industries. An electrical engineer by training, he launched the company with two co founders at a Denny's restaurant. It now earns $26 billion in annual revenue and employs 26,000 people around the world. Its technology is quite literally propelling us into the future. Jensen spoke with HBR Editor in Chief Adi Ignatius at our Future of Business virtual conference last week about what's new in AI and computing and the kind of corporate culture and leadership that keeps Nvidia agile and innovative. Here's their conversation.

Jensen, thank you for joining us. What I know is an early morning for you, probably a busy day for you. We really appreciate you making the time.

I'm happy to be here.

So Nvidia is producing technology that is quite literally driving the future. And I'm talking about the chips and processors that are driving innovation in AI and generative AI. I'd love your view as a sort of hardware guy. What's your thought on AI on gen AI? Do you think it will revolutionize how we work and play, or is it simply too soon to tell?

Well, at the core of generative AI is the ability for software to understand the meaning of data. It can understand the meaning of words, of course, letters and words and sentences and paragraphs and finding relationships and patterns. And from a lot of examples, it figures out, learns the representation of the data and even understand its meaning. So it understands the structure, it understands how it's constructed and the meaning of the information. This applies to English, it applies to pixels, 3D objects, proteins, chemicals. We've now used this deep learning method to learn the representation of a whole lot of different types of data, any data with structure. And so there are a lot of things in the physical world, in the world that we live in, that has structure. Once we can understand the meaning of the data and we can associate one modality to another modality. For example, we can associate the pixels of a cat in an image with the word cat. Once you could do this for one modality to another modality, then you can translate things. And so you could translate from English to Chinese, Chinese to French. You can also of course, translate from English to pixels, which is generative AI, or from pixels to language, which is captioning. Or you could go from text to text, summarization, so on, so forth. This basic model, the model that has learned representation of data, is incredibly powerful. And so now what the AI industry, what the computer industry is doing is mixing all of these type of generative methods and translation methods into all kinds of interesting applications. So I think at the core of this excitement is the fundamental ability to learn representation, learn meaning, and to go from one modality to another. And that's just incredibly powerful.

I'm guessing that our viewers by and large are familiar with gen AI, maybe are playing around with it. Probably relatively few are using it or applying it other than through experimentation. You know, I'd love. Do you have examples maybe, maybe using Nvidia technology of applications that you're amazed by that you're seeing out there already?

Well, we're using AI to design our chips, to find optimal placements, to help us connect transistors into circuits and help us optimize designs at a scale that, that isn't really possible before. We're starting to use AI to help us determine from a failure where the bug likely started from which line of code was responsible for the bug. And so we're using generative AI to help us from everything from designing the chip, optimizing the chip, to understanding where the failures are. One of the most powerful things that's happening, of course, is that generative AI predicts the next word or predicting the next letter, next word, next sentence, next paragraph, predicts the next pixel. It predicts the next frame of a video. These predictions could, could hallucinate. And one of the, the really effective ways is to. One of them, of course, is providing a lot of context, a lot of prompts that people do to, to set the background precondition, the model before you prompt it, before you ask it a question or it's before you ask it to do something. One of the things that's, that's really useful is retrieval augmentation. What that means is a database that, whether it's structured or unstructured or tabular or otherwise, that has Been vectorized. Vectorizing, or embedding a database requires you to learn the relationship between all the data among each other, just like you're learning the representation of the data that I was talking about before. And by learning the representation and learning the vectorizing the database, you could understand its meaning. What's really exciting now is that you can vectorize a database. You can connect it to a large language model, and that large language model essentially allows you to talk to your data. Every company has a ton of data in their company. Most of that data is now dormant. It's hard to use. Sometimes you have to do a lot of queries on it. But now you could have a database that actually understands meaning. And so you could do a semantic search into the database. It brings back out the information that you meant to query. And you take that embedding and you augment that with the prompt that you're prompting the model with. And so now the context, the background information with this prompt is much richer. So the thing that is really quite generalized now is the ease of being able to interact with applications because of large language models that understands your meaning. And then secondarily, all of these databases that you can now vectorize, which could then be used to augment the prompts and the queries. And so the applications across your company is really quite amazing. One of the fun things, of course, if you're a company with a large customer service department and customer service agents are inputting their interactions with customers, whether it's problems or complaints or whatever it is help that you're routing the customer through, all of that is somehow captured in a very large database. But wouldn't it be amazing if now that database was vectorized and you can just talk to the database? You know, what are the things that people are most upset about? What are some trends that we're starting to see? If we wanted to improve the customer service of our company? What are the top two or three things that you can do? You could talk to your database like you're talking to a person, and that's really powerful.

So I want to shift here a little bit and talk about some of the challenges of running a tech company in 2023. And, you know, for starters, how do you keep up with constantly shifting demand for. For new products, for new services? And how do you. How do you build a management team that can imagine the future and keep pace with so much change?

Well, you got to learn, you know, first of all, we're surrounded by people who are expert in the art, in the Fundamental and the foundational parts of computer science. And they could be, you know, compiler experts, or they could be chip design experts, or they could be processor design experts, or interconnect experts, or so on and so forth. And, and so you've got experts all over your company and they're the best in the field of what they do. Meanwhile, you want your company to be as alert as possible. And so the ability for information to be moving freely in your organization without being trapped in silos is, is something that's really important as the company gets larger and larger. Especially when the, when the industry is moving so fast. You want to be alert of very weak signals that could be coming from somewhere. And so it could be a research paper that's coming out of left field. It could be related to digital biology, it could be related to something in robotics, and somehow that robotics information, it could be generalized in such a way that it could be useful for how we train a chatbot. Maybe reinforcement learning was something that was really invented for articulating and animating stick figures and articulating fingers and things like that and playing a game. But it turns out that reinforcement learning, human feedback could be quite useful in fine tuning a large language model. And so you want to be able to see developments in adjacent fields or even unrelated fields and be able to somehow distill it back down to first principles and then extrapolate to useful applications in other domains. That fundamental ability of starting from really good foundation expertise in the company, the ability to stay alert of weak signals and, and then being able to go back to first principles. You know, when, when you make some observation, go back to first principles on why is that important and what is the reason, what are the governing dynamics associated with that, that phenomenon, and then be able to generalize them.

And then you have the political issues that affect your business. I'd love to hear you talk about how Nvidia is coping with the loss of. You know, they get billions of dollars in sales to China partly as a result of US Restrictions on sales of sophisticated chips to China.

Well, the restriction is a capability restriction. It's not an absolute restriction. The limitations are related to the performance levels of our GPUs. This is like restrictions of CPUs and computers that has gone back a very, very long time. The restrictions limit the level of capability or the amount of computation that a processor can do. So the first thing that we have to do is comply with the regulation, understand what the limits are, and to the best of our ability, offer products in the marketplace to serve customers that can still be competitive while complying with the restrictions. And so the challenge there is that the local markets, especially in China, has a lot of competitors. And so we have to compete in the marketplace, you know, while of course complying with the regulations that are, that are inspired by, by national security. It's not easy and it's a great challenge. And the competitors are moving quickly. And so it's like anything else, you've got to stay alert and do the best you can.

So it sounds like you're saying you're not giving up on that market, you're finding ways within what's allowable, finding ways to, if I have you right, create new products for, or new approaches to that market. Is that fair?

Only if the new products continues to be attractive to customers. The first thing that we have to do is understand what the regulation limits. And second thing we do is go propose products to the marketplace that customers might still find attractive. But there's no guarantees. I mean, competitors are coming up with all kinds of ways to attract the customer. And so we'll float the best possible products that we can that are fully compliant with regulations and take it from there. See if customers are still attracted to the products that we offer.

Okay, I want to go to an audience question now. This is from Steven. I don't know where Stephen is, but question is, do you think that AI will always need human supervision?

Human in the loop is pretty important. As you know, in robotics you could have different levels of autonomy. Most of the levels of autonomy, until you have a great deal of safety in all kinds of different ways. So for example, in the case of self driving car, if you want the self driving car to be able to take your family while they're sleeping in the backseat fully autonomously, you want to be fairly assured that the sensor system, the computing system, the vehicle control, all of it has diversity and redundancy. So for example, you might insist on making sure that you have cameras as well as radars as well as lidars. You might even have redundant steering control and redundant braking dynamics and so on, so forth. And these things give you assurance that irrespective of the conditions, the critical components that affects the functionality has diversity and redundancy. Diversity and redundancy is an important part of autonomous intelligence. In a lot of ways, even in companies, we have diversity and redundancy. You know, notice we talk a lot about diversity in companies because it allows the company to be more resilient. We have redundant ways of doing the same thing or, or trying to Analyze the same problem from different perspectives. Maybe the, the sales organization is augmented by the marketing organization, which is augmented by, you know, so on and so forth and different, different types of, of ways to ensure the vibrancy and the resilience of the company. And most autonomous systems, whether it's a robotic car or it's a large organization, wants to be designed in such a way that you have diversity and redundancy. And then above that, you want human in the loop. Sometimes you get to a level where you don't believe you need human in the loop inside the car. Maybe you have human in the loop outside the car and you're monitoring, monitoring the effectiveness. This is no different than air traffic control. For example, you have two pilots. You also have redundant autopilot systems, but you also have air traffic control and pilots keep an eye out for each other. And so there's a lot of different ways that autonomous systems are designed for maximum safety and reliability. The same line of thinking will have to be done for generative AI. We have to invent the technologies for safety. We have to apply engineering methods that allows us to design systems that are safe. Lastly, for quite a long time, we will surely have human in loop. For example, while you're learning the next batch from the next batch of data before you release it out into the world, you have humans validate it and test it and red team it instead of just have it be continuously learning and continuously updating all by itself. And so there are a lot of different methods that we ensure safety and human in the loop is one of the best ways.

I'm interested in hearing more about Nvidia's culture. One of your vice presidents was quoted as saying, working at Nvidia feels existential every day. We feel like the company is going to go bankrupt tomorrow. So I'm wondering, is that deliberate? Do you try to instill a sense of paranoia to try to spur innovation?

You, you don't have to instill it. If you don't think you're in peril, it's probably because you have your head in the sand. There are no, no companies that are, that are assured survival. We have the benefit of having built the company from the ground up. And so I. And having not exaggerated circumstances of nearly going out of business, actual experiences of nearly going out of business a handful of times. We don't have to pretend that the company's always in peril. The company's always in peril, and we feel it. I don't need, and the company doesn't need assurance that we will, we will do well in order to do our best work, you know, and so I, I think the company living somewhere between aspiration and desperation and living somewhere within that, you know, within that spectrum is a lot better than, than either always optimistic or always pessimistic. And so I, I think a, a guarded level of optimism and realistic understanding that, that the company could be in peril at any given time and we have to stay focused and alert and do our best work and constantly earn the right to be in business. Those feelings and that sensibility I think is really good and healthy for companies to have.

From what I've read, your management style is somewhat unusual. That you maintain a very flat organizational structure, that you have very few one on one meetings. Talk about your theory of management.

Well, an architecture is designed for the company to do its work, to operate as efficiently as possible. And one of the things that is really important for a company in technology that's moving as quickly as it is. We invented this technology called accelerated computing. Whereas Moore's Law in the past, it's no longer nearly this fast. But Moore's Law in the older days would increase performance by a factor of 10 every five years and a factor of 100 every 10 years. But in the case of accelerated computing, we're moving somewhere between a thousand to 100,000 times every 10 years. And so when you're moving that fast, you want to make sure that information is flowing through the company as quickly as possible. You want the company to be as aligned as possible. And also our company builds really complicated things. People think our GPUs are the add in cards that we design for gamers. And, and they are, we, you know, we, we build those with great pride and great Joy. But the GPUs that we build for AI weigh 70 pounds. They have 35,000 parts. Out of those 35,000 parts, eight of them come from TSMC. They consume 10,000amps. Obviously they're so heavy you use robots to, to manufacture them. The computers are so advanced that it takes a supercomputer to test another supercomputer. And so these GPUs that we're building are insanely complex and it requires a lot of software to run it. And for a company like us that designs the fundamental chips, creates the systems, create the software, and creates all the networking and all the silicon photonics and connected into a large data center and help companies stand it up and operate it, our company information has to really move quickly through the company and, and really, really have no Barriers and no boundaries. And so we architected a company that allows us to be able to work across this really complicated stack of computing technology on the one hand, build incredibly complicated things on the other hand, and be very comfortable moving at light speed. If you want a company like that, then the last thing you want is for it to be information to, to pass along hierarchically. And if you look at most organizations, there are only two or three people at the top. And if that starts that way, then the number of layers of management, layers of managers before you get to somebody who's actually dealing with ground truth could be 7, 8, 9, you know, 10 layers. And I didn't want that to happen. And so the best way to do that is to put the most at the highest level to be as flat as possible up there. And so we probably reduced, you know, three, three or four layers of out of our company just by doing so.

Yeah. So we have time for one more quick question. I wanted to bring in one more audience question. This is from Leon. And the question is, what can you say about quantum computing? To what extent are you investing in it and what can you say about that?

Well, in order to build the world's next fast computer, you need the world's fastest computer to help you design it, simulate it, create new algorithms for it. And that the world's fastest computer is Nvidia's accelerated computing systems. And so we don't build quantum computers, but we work with just about everybody in the quantum computing ecosystem, from people who are doing the basic quantum computing design, to algorithm developers, to architects who are working on classical quantum computers architectures. And so we work with just about everybody out there. The quantum computer won't come along for, in a real industrial way for probably another decade and maybe two. And when it gets here, it'll likely be an accelerator that's connected to the Nvidia accelerated computing systems. And some of the computation will be done quantum computers. A lot of, a lot of the computation will be done on classical computers doing pre processing, post processing, and so on and so forth. And so I'm looking forward to when quantum computers are going to be useful and effective. And when it comes, classical computers, and particularly accelerated computing and AI will be a very big part of it.

That's Jantin Huang, the CEO of Nvidia, speaking with HBR's editor in chief, Adi Ignatius, at our Future of Business conference last week. And we have more episodes and more podcasts to help you manage your team, your organization and your career. Find them at hbr.org podcasts or search HBR in Apple Podcasts, Spotify or wherever you listen. Thanks to our team, Senior Producer Mary Du, Associate Producer Hannah Bates, Audio Product Manager Ian Fox, and Senior Production Specialist Rob Eckhart. And thanks for listening to the HBR IdeaCast. Be back with a new episode on Tuesday. I'm Alison Beard.