The Man Who Wrote the Book on NVIDIA

Hey, Upstream listeners! Today, Tae Kim, author of The NVIDIA Way, joins Byrne Hobart and I for a look inside one of tech's most influential companies. In the conversation ahead, we discuss how CEO Jensen Huang transformed a scrappy graphics chip startup into tech's most valuable company. Tae brings us inside NVIDIA's legendary culture of excellence and shares fascinating insights about Jensen's leadership that few have heard before. Please enjoy!

Hello everybody. We're lucky to have a very special guest on this episode, Tae Kim. Tae, welcome to the podcast.

Thanks for having me, Erik. Great to be here.

Byron, do you want to give some context on why we, why we decided to have Tae on? Yeah, yeah, absolutely. So, so Tae wrote a book called The NVIDIA Way. It's really good. I just read it. I recommend that other people read it. And it is just like, it's a look at what is an incredibly important company that, as you mentioned at the beginning of the book, it sounds like you were kind of shocked that nobody had done an NVIDIA book. And that's kind of weird. Like we use products that are built on their products all the time. And even if we don't use them directly, like we are, we are all beneficiaries of the flops that NVIDIA produces at great scale and at surprisingly low cost. And yeah, I find it a really fun book because it feels like sometimes when you read a book about a company, you do find that they had some secrets, some concept, some big idea that they were able to execute on. And you look at a company like Google, and it's like at every step, the question is always, where is the world's unorganized information? How do we organize it? How do we make it accessible? And then how do we put ads into it? That's a really powerful insight that can show up just about everywhere. Or you could look at Microsoft's early insight of, hey, computer hardware is really expensive. It was really expensive when they started. It's only going to get cheaper, and therefore you want to own the software, which is this complement to it, where every time there's a new iteration, not only is the hardware cheaper, but now there are more users. And if your software has almost no marginal cost and the hardware does have a marginal cost, you want to be selling the near-zero marginal cost thing and let everyone else do the annoying hardware stuff. And then Nvidia does the annoying hardware stuff. They are the ones who are actually connecting wires to things and figuring out how many watts this thing needs and how much heat it throws off and all of that stuff. And so it felt like part of your thesis in the book seems to be like, there isn't really a secret. Nvidia hired great people, has an amazing founder, and they just work really hard and set incredibly high standards. And it seems like they're just always on. You have this anecdote about how Jensen, the way he unwinds on the weekends is that he pours himself a glass of a really nice scotch and then just answers work emails for hours. Yeah. So tell me a little bit about just, I guess this somewhat rehashes some of the early parts of the book. What was the genesis of this? And how did you think about the process as you were writing this book? Were you looking for the secret and didn't find it? Or was it like you pretty much had this impression that, no, They just work really hard and they're smart. And so that hard work has a lot of leverage.

So the genesis was I got a cold email from a publisher in May of last year, and a friend of mine, Matthew Ball, who wrote the Metaverse book, suggested to his publisher that Tay could write the book on Nvidia. My first thought was, there's no way no one has written a book on Nvidia. I mean, this is a little bit before the AI stuff really took off. But it was the largest chipmaker by market cap. As a PC gamer, they've pretty much dominated the graphics card market for video games for decades. So I went to Amazon and took a look and I couldn't believe it. No one has written a book on Nvidia. And there's 4 or 5 books on Facebook. There's multiple books on Amazon, Apple. And I just thought for a sec, I could do this. Like, I know the technology, being a PC gaming nerd and addict for the last 30 years. I've been in media for the last 10 years, so I've been figuring out how to become a better writer. And I know the business because I've been on Wall Street for a bunch of years as a technology analyst, business strategy. So I thought I had the 3 components to really do a good job. So quickly, within a few weeks, I met the publisher, got a book agent, and we're off and running. And the first thing I did was call— LinkedIn is an amazing resource for For an author and, you know, just reaching out to NVIDIA employees. And within 6 to 7 calls, I realized the big thing that differentiates NVIDIA, like you said, was the culture. It's completely different from other companies, the large technology companies. Every former NVIDIA employee that goes to a large tech company like Google and Microsoft says it's extremely difficult to adjust to the culture. NVIDIA is faster. The work ethic is, is on a different level. You mentioned something about the secret sauce. There's this one other anecdote about when they acquired the assets and hired 100 engineers from 3dfx. And 3dfx was the number one graphics player in the mid to late '90s, and they got beat up by Nvidia and pretty much destroyed. And the 3dfx engineers came to Nvidia and were looking for the secret. What's the secret? What's the magic trick? And then they realized the only secret is you just have to work Crazy and relentless execution and the work hours. It was just working harder. So that, I think, when you look at the Nvidia way, I think the work ethic from the beginning and even today, you mentioned Jensen working on Sunday night. He loves to work. He literally says it's relaxing to him. When he goes to movies, he doesn't remember the movie because he's constantly thinking about work. So that Relentless kind of hard work ethic is a key component. The second component is cultivating talent, and we could go into this later, but he's constantly paying his superstar engineers. If they're doing a great job, he's going to cultivate and retain that talent by doubling, tripling their stock comp. And he's constantly looking to hire the best people. So when he interviews people, he's finding out who the best person is. At their former jobs. And the last thing is just speed and velocity in doing everything. I think large companies, when you go to large organizations, the internal politics just take over and everyone's looking out for their own power, backstabbing, taking credit. So when there's new projects, new disruptive projects, they just get stuck in the quicksand where you have to get approvals from 5 different leaders. For it to go forward. NVIDIA is the exact opposite. NVIDIA, Jensen just gets the top 10, 15 people in a room, including the lower-level employees, and they figure out what's the main problem, what's the main thing to discuss, hash it out in an intellectually honest way, and then they move and execute that vision. So that speed and velocity, I think they call it speed of light at NVIDIA, is a huge component of why they've been able to succeed.

Yeah.

Yeah.

Well, I do want to get to the world's weirdest org chart in a bit, but I'm also like, one of the things you pointed out that was really striking to me because it just hadn't occurred to me, but it is of course true is Jensen's actually one of the longest tenured CEOs of any publicly traded company. Like you have Buffett at Berkshire and then I think you mentioned one other person, but it's like pretty much he's it. And, you know, was, has been running NVIDIA longer than Gates ran Microsoft and longer than Larry Ellison was CEO of Oracle. still calls a lot of the shots. So, yeah, and there's obviously going to be a selection effect where, yeah, he has great performance and a really long tenure because if his performance was really bad, at some point the board or an activist investor would have told them to find someone else. But, yeah, what do you make of that? How does someone have that kind of endurance? And I guess the follow-up question is, if you have someone like that who has been— he was there day one, naming the company and, you know, they're like in the Denny's booth figuring out what the initial business plan would be. It's still there today, you know, on stage and doing all these events and things. Who comes next? And does NVIDIA have to have a different kind of structure and approach once there's someone else in charge?

I think part of it, like you said, is he's just, he's been so successful across the decades. So he built up a level of credibility that No one can take away from. I mean, right now they're on top of the world. It's like unprecedented, the level of revenue. I think 6 quarters ago they were doing $7 billion in revenue and now they're doing $35 billion in revenue a quarter. This is hardware and chips. This is not software. You could just copy the bits over and over again. So you have this level of success. I think he just loved, like I said, he loves to work. This is his passion. Nvidia is life. He says, I love Nvidia over and over again. So if you have that combination where he loves his job, he works harder than anyone else. He talks about how I might not be the smartest person in the world, but no one's ever going to outwork me. Like he just has that engine inside him that is unbeatable. So you combine those 2 things and the level of success he had, no one's going to question and no one's going to tell Jensen what to do. There was an activist fund, Starboard, which is pretty famous, that almost did something, you know, My rational kind of speculation talking about covering that in the book. But lucky for Jensen and Nvidia, they just did a big stock buyback. The stock popped 20% and Jeff decided to move on. Maybe history's a little different if they, you know, the stock didn't go up and, you know, he took the gain. So there were times when Wall Street was all over it when the gross margins were coming down for, and this is not just 1 or 2 quarters, this is like 4 or 5 years of weak gross margins because he was investing in the CUDA cores on the GPUs. So I mean, a lot of it is kind of maybe happy accidents, but over time, like, you know, he's proven to be able to see beyond corners of all these big tech, all along the way. So yeah, that's where you have it. In terms of future, what happens after Jensen, the first thing I answer to that is He's in his early 60s. He loves his job more than anything. He's working harder and just is doing better than ever in the last 30 years. I just don't see him leaving anytime soon. I mean, someone mentioned Morris Chang, like, worked till he was in his 80s. So there's still many, many years ahead. When he does eventually leave, the big question is who the next CEO, the competence of the next CEO. And I think history has shown That it could be good or bad. I mean, you had John Sculley was a disaster, almost drove Apple to bankruptcy. Steve Ballmer, you know, wasn't that great, even though the financials are better. I think, you know, he made a lot of mistakes. I do.

I do think Ballmer's underrated, but it is definitely true that Gates was just a very special CEO.

I think it's, yeah, when you, if you compare Gates to Ballmer, I mean, Gates is on a different level in terms of knowledge of technology. I mean, you could argue, you know, Ballmer did a good job, sort of like Tim Cook, like in many managing the financial aspects in the business. But yeah, that's the big question. Are you going to get like a triple, quadruple-A CEO again, or are you going to get someone that's not so good? And corporate America is full of CEOs that are pretty— I'll just say it, they're pretty incompetent. And like I said, as of now, I don't see Jensen leaving anytime soon. Is there an obvious successor at Nvidia? No. I mean, I don't see anyone that's obviously up to his level, but yeah, but we'll see. You never know.

And I think one of the other things that makes that long tenure really impressive is that Nvidia is in a cyclical business. And it feels like a lot of maybe the marginal buyer of Nvidia stock right now is maybe less aware of that than someone who's been following the company for a long time. But it is the case that sometimes there are a lot of GPUs just gathering dust somewhere and not selling as fast as expected. In fact, it hasn't been that long since Nvidia had to take a big write-down. So, you know, how do you think about how they navigated that cycle, especially because it does feel like the vibe in the book is that Jensen is always saying, you know, we won last time, so let's bet everything on this next thing. And that seems like a tough, you know, definitely if you, if you get that kind of call right a couple cycles in a row, it does really well, but If there is a long, an extended difficult cycle, not so good. So yeah, how do you think about them, how they navigated that cyclical, like the combination of like, it's a secular growth story that, yeah, we will need more GPUs over time, but in any given year, you don't know if we need, you know, you know, 50% more or like 20% less than the year before.

So, so this is a nuanced situation and answer. I mean, I agree with you in the past, they've been very cyclical. Sometimes the product, I think Turing. And NV30, they have a couple really disastrous, not so great products where a competitor does designs a better chip than them. So there's always that risk. And you know, there, there may be once every 3, 4 GPUs, they just don't make a great product. There, there was this, the crypto stuff, which I actually didn't have time to cover in the book. And I actually, in my day job with Barron's, I was very out in front kind of predicting that the problems of crypto and Ethereum when they, when they changed the The way Ethereum worked. I do think the AI stuff is a little different because this is the early stages of a massive computing shift. When you have the interim CEO of Intel saying this is one of the biggest computing shifts in decades from CPUs to GPUs, I mean, Intel is the dominant player in CPUs and the interim co-CEO of Intel is saying, This is a really big shift, the biggest in decades. That to me tells me like this is going to be as big as mobile when Apple did the iPhone, as big as Wintel when Microsoft and Intel just rode the PC revolution for decades. So I do think the cycle here is less of an issue. It's really a massive computing shift. Where we're going from the CPU stuff, which is very static retrieval based on files, right? Windows Server, you try to look for one file to this GPU computing paradigm where it's a completely different paradigm. You're basically ingesting all the information you have, distilling it into an AI model, and be able for that AI model to generate a business intelligence insight or generate a new piece of content. It's completely different. So I do think we're still in the early innings of this wave, computing shift wave that is going to last for several years. But that, I mean, if you look at the chart of Nvidia, the data center revenue didn't really take off till like early last year. Like the vast majority of its revenue was gaming GPUs, like 50 to 80% from the vast majority of its life. So it's only in the last 6, 7 quarters that That the data center part of their business has really taken off.

We'll continue our interview in a moment after a word from our sponsors. Most of you probably know how much the Turpentine team loves Notion, the sponsor of today's episode. We use it every day for editorial planning, transcript markups, project managing, knowledge sharing, and it's where I drop random 3 AM ideas that turn into episodes. Now it has the power of AI built right inside of Notion and woven across our entire workspace. It's been a total game changer for our productivity and focus. We just used it the other day to architect the content schedule for next month across our network. Notion combines your notes, docs, and projects into one space that's simple and beautifully designed. The fully integrated Notion AI helps you work faster, write better, and think bigger, doing tasks that normally take you hours in just seconds. Try Notion for free when you go to notion.com/upstreampod. That's all lowercase letters, notion.com/upstreampod, to try the powerful, easy-to-use Notion AI today. And when you use our link, you're supporting our show. That's notion.com/upstreampod.

Yeah.

And, you know, I really liked the bits that you wrote about this transition from NVIDIA as a graphics card company where their core market is gamers who want to play the latest games. Maximum resolution, full detail. They want the fog effects and the lighting and the explosions need to look good, et cetera. Somehow there's a transition from that to we're not modeling this fog in order to make Silent Hill look even creepier. We're modeling fog because we work at a major lab and we're doing some kind of really elaborate weather modeling thing. And I liked that very early on, it was basically people tricking the GPU into thinking of something as, like, pretend you're drawing a triangle and then solve this scientific computing problem for us. And that almost felt a lot like just the way that sometimes human cognition works, where you look at a string, a series of numbers, and you don't see a pattern, and then you make a graph and you realize, oh, this is trending up or trending down, or it's getting more or less volatile, or why is there this weird blip here? Yeah. So talk to us a little bit about how that transition worked. And I'm especially— and this kind of shades into the other question of just how aggressive Nvidia is as a business. How did Nvidia navigate this, especially when they had products that were meant for these data center type scientific computing workloads. And then they also have just off the shelf, much cheaper GPUs that were designed for graphics, but could be used for other things that a lot of researchers just preferred to buy because they were cheaper.

So you touch upon a great point that a lot of the AI stuff, all the high performance scientific research, it's matrix multiplications, the same kind of math and all that calculations and And math is the same math that was in video games. Like, I think I mentioned in the book how for video games, you have to simulate the physics of a bullet, like how it comes out of the gun, where it lands, and all that kind of high-performance, intense computing simulation is the same stuff you do with science. You know, figuring out how water moves inside of fluid dynamics of a cloud, you know, trying to predict how biological molecules would work. For potential drugs. So all this is kind of simulating real-life physics, and that's how Nvidia was able to move from video games to CUDA and all the AI stuff later on. It's basically the same kind of math. It's the same kind of physics. So that really helped Nvidia. The funny thing is everything Nvidia has learned in the past in terms of business, it's the same playbook that They won with video games, they're doing to win in data center AI. I touch upon how in the late 1990s, Jensen figured out how all these graphics card companies would dominate for one cycle and then lose their perch. And he learned that the problem was that a typical chip took 18 months to develop and ship. And then a different graphics company will come out with a with a better chip because their 18-month cycle was up. How about I just change that to 6 months?

Yeah, I always like that, like the, because it feels like maybe, you know, what if someone at 3dfx has the same observation of, hey, our product cycle is 18 months, PC product cycle is 6 months. And, you know, I think for a lot of people, they would say, therefore our business is lumpy and we're not going to be number one in 2 consecutive years. And then the NVIDIA answer is, Hey, all we have to do is work 3 times as hard and then we'll, we'll always have something new whenever there's a new PC cycle. And it seems like that, that worked out well. Like, do you think that was, that was literally it was that the, the chip comp— the competing companies just didn't either didn't have the, the capital to do that or just did not have the, the will to work that hard?

I think there are 2 components. I mean, I'm being a little simplistic when I say, oh, just go from 18 months to 6 months. I mean, it was an 18-month architecture, but they They created 2 derivatives that were faster. It wasn't like a completely different architecture every 6 months. But I think there are 2 components of why NVIDIA was able to do it. One is just technical expertise. One of the co-founders, Kurzweil, was able to do this mini kind of virtualized operating system that allowed them to, if they couldn't make a new great feature in hardware fast enough, you could Virtualize that into software and vice versa. So that gave them a lot more flexibility than the competitors that didn't have this virtualized operating system where you could put features in software or hardware. So, and then it's just the relentless drive of Jensen. Like he, like, as we, as, as I mentioned in the book, he is very, very tough on his employees, especially if they don't come through on the schedules that he wants. So he will dress that person down. The one anecdote, one Best anecdote I have is during one company meeting in front of the entire company, he dressed down this manager of the Tiger 3 chip. He was late and he just kept on getting the camera in his face like, Rayfield, you need to finish this chip. You're not doing a good job, and just humiliate him in front of the entire company. That might not be great. That might not feel great, but it definitely gets people to work really hard. Like, like you don't want to be that guy in front of the whole company being dressed down. So it's a combination of technical expertise and just that hard-driving culture that's driven by Jensen. So that's why they're able to do that 6 months instead of 18 months. And that same playbook is being done today with AI. And last October, over a year ago, he said the data center AI GPUs were going to accelerate the product cadence. from 2 years to every year. So if you're AMD, that's really hard. You're behind already, but now you have to match Nvidia's new product cadence. And the other part, not just the hardware, is Nvidia from the beginning has invested tremendous amounts of money on software, their software drivers. And from the beginning, they sent their engineers to all the video game publishers to make sure their software is working correctly. It's a strategy they call devs before revs, developers before revenue. So Jensen always knew from the beginning, it's really important to make sure your software is the most reliable, the best, the most feature-rich, and he's willing to invest tons of money, even though it directly hurt their P&L statement. But he knew the secret sauce for their competitive advantage was going to be this, was going to be the software. That's exactly the same thing that's happening with AI right now.

Yeah, that was actually going to be my next question. I noticed there is this thread throughout the book, I guess, starting maybe a third of the way in with that point about how they started just releasing their own, releasing drivers directly to end users on a monthly cadence rather than just hoping that nobody would ever ship any software that would break something. And so you have that, and then you have the virtualization thing where if there's a function that the GPU can't handle in hardware, you can emulate the hardware and That it's slower, but it's still doing— it's still full featured. And then there's, of course, CUDA. So it does feel like Nvidia has just made this gradual transition towards being more of— they're still selling hardware, but more of their advantage, where their lock-in comes from software and the fact that people have learned these software libraries and are used to being able to trust Nvidia software. So which kind of feels like that has to be a tough transition for a hardware company. It has, because they feel like their competitive advantage is hardware, and then They're not really monetizing the software directly and they are making this investment that pays off over a really long period. So yeah, did you get the sense that that was actually a difficult transition for them to make, or was it just like, if you have high enough standards for your product, then you'll solve all of these adjacent problems because that's the only way to ship something that's actually satisfactory?

It wasn't really a transition. It was from the beginning, Jensen and the co-founders knew the importance of software from, from the prior experiences at Sun Microsystems for the 2 other co-founders. So it was there from the beginning. They knew over time That was that was so important for customers. One of the coolest things after this during this book process was having John Carmack tweet about my book. He he's the creator of the two major PC gaming kind of revolutions in the early '90s. Everyone that that age knows of Doom and Quake. I mean, those are the two most important PC games in history. And he he said he gave a historical context as he was reading my book, saying that the importance of unified drivers by Nvidia, like you said. was so important because typically before that, you had to download the drivers from the PC graphics card website, and then you would have to download one driver for your graphics card model. It was a convoluted, complicated process. And Carmack said most of these graphics drivers were terrible. They were buggy, had performance problems. So when Nvidia went with the unified graphics model where you just downloaded one file, It worked with all the NVIDIA graphics cards and NVIDIA wrote the reliable cards instead of, you know, 5 different PC makers and 5, you know, 40 different graphics card makers writing one, creating one file for one particular GPU model. It just made things work. Like it was a huge revolution in how it all worked. And John said, John Carmack said this was a huge deal for the development of the PC gaming industry. It was a huge advantage for Nvidia for making these drivers. And the same thing, like I said, the same thing is with CUDA. The reason why AI startups and companies go with Nvidia's AI CUDA platform is that the software works. It's reliable. You don't want to save 30% on the chips, but then you put the fate of your business on a platform that has tons of bugs and has reliability issues. You don't want to do that. SemiAnalysis just literally wrote a piece talking about how they had no problems trying to build their AI stuff and test on NVIDIA CUDA, and then they tried to do the same thing on AMD software and there was nothing but problems. So if you're a company that the fate of your business depends on keeping up with AI when your competitor's working on an AI model or a feature that's using NVIDIA, you're going to use NVIDIA too, because you can't risk the future of your business on something not working.

Yeah. And one of the interesting things that you mention in this is, in the book, is NVIDIA's strategy of selling into academic environments where they would do a grant and it would take the form, you know, partly cash, partly just hardware and then people would learn to build their, they'd learn to do their simulations on NVIDIA hardware. And I, through the newsletter, I do some recruiting and I talk to people who are getting PhDs in STEM topics. And it feels like half of the PhDs, it's basically, it's mostly ML ops. Like it's mostly how do you get enough time on, on a limited set of GPUs and how do you negotiate that? And how do you optimize to, to minimize the training cost of whatever your training. So yeah, tell us a little bit about how that strategy worked and where you think the payoff showed up.

So they did this for a long time. I talked to David Kirk, who's one of the early chief scientists at NVIDIA, and they said they donated graphics cards, they donated GPU systems to all the top universities that They had graphics programming courses and programs and degrees, and they just wanted the students to build their expertise on NVIDIA GPUs. And eventually all these students will become engineers at different technology companies, and they're going to use what they're used to, which is NVIDIA. Same thing happened with CUDA. David— the funny story is David Kirk wanted to get CUDA incorporated into all the top PhD programs and all the computer science programs. And he traveled the world and did over 100 talks and speeches to the top people at universities. And he had no success whatsoever until the University of Illinois finally told him, why don't you teach the class? Why don't you do it? He's like, no way. You know, I live in Colorado. I have a good life now. And then he said, no, no, no, no, we'll match you up with one of our top professors and you should do it if you really believe this is the future and you really want to do it. So he did it. He literally flew every other week from Colorado to University of Illinois. They created a CUDA textbook and that was the genesis of this CUDA program that got copied by hundreds of colleges and universities. Other companies don't do this. We're going to put one of our top resources Have him teach a class at a university because it's going to be really important. And now, you know, CUDA is the standard that's being taught everywhere. So that, that kind of long-term thinking is just, you know, other companies don't do.

It does, it does kind of remind me of the, the Apple strategy of also selling into elementary schools, but I, I'm, I'm of the generation where when they were doing that, it was pre-OS X. So I sort of, my, my impression of the branding of Apple was Apple computers are good for goofing off when you should be doing your homework or should be studying. And, you know, they are like weird and are missing a lot of things that your Windows computer at home has. And, you know, you don't see, like in my case, my, my parents had PCs, so I could see work happening on the PC. I knew that I could play games on the PC and then Apple was this other, other thing and, you know, came in bright colors and stuff like that. This was like late '90s. So.

But it did work. I mean, I grew up with Apple IIs in the schools and a lot of people bought Apple because of that.

Yeah, I think it definitely worked. And then I guess part of the unique factor for NVIDIA was that people were doing pretty serious work, and then when they got into industry, they would suddenly have a much larger budget and they'd be making more money. And it was a, you basically get someone when it is actually pretty cost effective to really build the brand for them. And then that brand value is retained later on. I also do want to talk about, it seems like You know, a lot of this, a lot of the story is a story of technology and of people doing intensive research. And you have some great stories about how that process worked and some of the close calls and insights and things like that. But there's also the story of NVIDIA is just a business that definitely plays hardball when they can. And I think you talked about this with scientific computing where there were researchers who said that the consumer-grade graphics cards are perfectly fine. And then NVIDIA would try to say, no, they don't have the error correction that the enterprise card, which is, you know, Same thing, just much more expensive, has. And then of course you say this to someone who spends all of their time playing around with hardware and is trying to prove true things for a living, and they will just go out and test it and find that no, that's actually not true. And then more recently, it seems like Nvidia has maybe not played games with, but has, they've been very aware of the power that they have to allocate GPUs to particular customers. And I kind of wonder what percentage of GPUs would go to the OpenAI and Microsoft complex if It was just if NVIDIA were just selling to the high bidder, and then how much does NVIDIA benefit from having lots of GPUs in CoreWave and Lambda data centers that anyone can use and anyone can train their models on?

I think those are 2 different issues. So let me just touch on the CoreWave stuff. I actually think it's smart of NVIDIA to kind of spread the GPUs around. I mean, there's more demand than supply right now, and if you just give all your GPUs to the Amazon, Meta, Microsofts of the world, you're kind of concentrating that power. So it's actually creating a more competitive market by making sure the new startups, the new companies like CoreWeave are— and Oracle is actually a new entrant into the cloud market. So Nvidia kind of spreading the GPUs around is creating a more competitive market. And obviously all these big tech companies like Microsoft, Amazon, and Google also have their own AI custom chips that compete with Nvidia. So it makes sense For Nvidia to make sure all the power isn't concentrated in the top 3 largest companies in the world. So I think that's one aspect. The second aspect is I talk about how Nvidia's the one chip company that always wants to raise prices because they're adding value. And one executive told me that they're a profit-making business. Their point is to make more money. But with that, By making more money, they can do more research and development. They could innovate more and push the envelope, push the technology envelope more. So yes, it's a capitalist company. Their aim is to make more money, create shareholder value, make the employees well off financially. But there's a positive outflow of that is that they could do more research and development. They could do more things that help society. And there are a couple things that they're really excited about in terms of digital biology, potentially finding cures for cancer, and robotics. So yes, Nvidia wants to make more money and they'll do what it takes within the bounds of the law to do that. The other thing is they do have this really, I think, different business strategy which comes up called rough justice. Which is a Jensen business philosophy, which the executive Rick Tsai, who was NVIDIA's representative at TSMC, told me. And he eventually later, he became the CEO of TSMC and now runs MediaTek. So he's actually a really big figure in computing history. He told me when he first met Jensen, like within a year or two, they were having problems. He traveled to California and went to some restaurant that wasn't a great restaurant. He pointedly told me, and he said Jensen told him his philosophy of business was rough justice, which meant in any particular deal or interaction they had with a partner or supplier, maybe TSMC would get 60% of the benefit and Nvidia would get 40% of the benefit. And maybe another time Nvidia would get 60% of the benefit of this interaction. Because they needed it. We need this, we need some help, and TSMC will get 40%. It's just rough justice. Over time, if it's 50/50 over 3, 4 years, that'll be great. But we have to build this trusted relationship where it's like a friend. When I need you at this point, you're going to help me out, and then later it's going to be vice versa. And that's the way he saw business partnerships and relationships. And that's Rick Tsai thought that was so profound. Like no other business that he interacted with thought of it that way. And then that as you build that trust relationship where you're there for each other at different times, that's how you build a strong partnership. And that's, and he called TSMC and NVIDIA like one of the best partnerships ever in business history over 30 years. So it's not about just getting every dollar and penny of your partner or supplier at all points. It's about being able to build that trusted partnership and relationship over decades. So I think the hallmark value of Nvidia is long-term thinking. So if you just are always about getting that last dollar out of people, people won't want to deal with you over time. And Nvidia and Jensen have always thought about long-term. What's the long-term benefit for the company, the customer, and Nvidia. So that's why they've been able to just invest long-term, figure things out that other companies haven't been able to do, and think 5, 10, 15-year increments instead of the next quarter or the next year that most of the technology industry and Wall Street wants.

Yeah, I think that makes a lot of sense. And yeah, definitely the Nvidia-TSM partnership is just really interesting and unique because There are a lot of companies where they, they will partner together and it is mutually beneficial, but each company kind of makes sense on its own. And then with NVIDIA and TSMC, it does feel like the incremental fab probably didn't make sense without NVIDIA as an anchor customer. And then the incremental GPU doesn't make sense unless you're actually pretty sure that someone out there can manufacture it and that someone is almost certainly TSMC. So yeah, it becomes this, this really interesting mutual dependency. And it is like, it's, it's fun to look at the, the conference call, like earnings earnings call transcripts for both companies because Nvidia investors are generally very, very happy with the margin picture. And then TSMC, one of the big questions is always, why can't you just charge them more? Why is it that you are making this little markup? Like you have manufacturing business and for a manufacturing business, incredible gross margins. And yet as soon as that product leaves the fab and gets an Nvidia logo slapped on, it gets packaged up and Put in a box that says Nvidia on it, it's suddenly worth a whole lot more. And yeah, it does seem like maybe you know a big part of that has to be that Nvidia was there as a customer and was willing to willing to take a risk on on TSMC when they just didn't have the the brand name and franchise that they have today. So it is it is a nice story.

The conference calls are really funny because the TSMC executives always kind of joke without naming Nvidia. They They joke about NVIDIA's margins on the calls, but I mean, TSMC is doing great on the margin side too. I mean, compared to everyone else. So both are, both have, it's been a win-win partnership, especially these days.

Yeah. Yeah. And so like a lot of the story, it is the story of a company where they, they have a vision for what the product will be. They bring together the team that can actually realize that vision. They sell that product. And then you also talk about the, their, their M&A. And that they, they do the occasional strategic acquisition, like 3dfx had plenty of good, good designers and, you know, good, good engineers. And Nvidia was able to pick them off after 3dfx failed. And then there's the, the Mellanox deal, which I thought was a really interesting one because it just, it feels like there's this cohort of companies where they are very relentlessly product-focused companies with a really serious engineering culture. And then there's the cohort of companies where they like strategic acquisitions and they, they're happy to get mixed up in bidding wars. And, and then sometimes when those collide, it's like someone who is really good on the technology side, just kind of loses their head on the financial side and overpays for something. And then in this case, it turns out that they seem to have dramatically underpaid. So yeah, tell us a little bit about that transaction and some of the corporate drama there and the outcome. And I guess maybe the interesting counterfactual is if Mellanox had stayed independent, would it be as big? Would NVIDIA have tried to replace some of that internally, or would it just— would things have just not worked out as well as they have so far.

So Mellanox is a very important event in NVIDIA's timeline. They weren't seeking Mellanox. Mellanox kind of fell in their lap, which, like, as I mentioned with Starbird before, Starbird kind of forced the CEO, if you don't hit these marketing targets, we're going to kick you out. So the CEO decided, okay, forget this. I'm just going to sell the company. And Mellanox went up for auction. Jensen looked at it and after Very short period realized I really want this company. I think the world is going in a certain direction, and Mellanox is going to be critical to that future. So when they actually beat out Intel to buy Mellanox, if you look at the press release in 2019 when they announced the acquisition, you see Jensen kind of laying out this vision. I need Mellanox. Nvidia needs Mellanox because someday AI is going to work in these data centers. It's going to be spread out between tens of thousands of servers with GPUs. And Mellanox is going to be a critical factor on how that happens. That's exactly what happened. I mean, it's incredible. Like 3, 4 years later, that's exactly what's happening with these AI superclusters. And then you look at the amount of revenue they're generating. They spent like $6 or $7 billion for Mellanox, and now the run rate on a revenue basis is like $13 billion a year growing at some ridiculous number. So it literally was one of the best acquisitions in history because it really enabled Nvidia to create that whole data center kind of full-stack solution with software, hardware, the networking, just pulling that all together like Apple does with the iPhone and offering that complete solution to their customers. But I do think you touched on a good point is these aren't acquisitions where you're trying to win a part of the supply chain. Like 3dfx did disastrously with STB, was a critical M&A deal for them. This is about technical capability and talent. And Mellanox were the best in the world in InfiniBand and this low-latency scalable networking technology. And NVIDIA was able to get that, incorporate it, and sell a broader, more full-stack product in the end. But they, I mean, throughout their history, other than being very aggressive in hiring engineers when they become available, they haven't done any big M&A, that much M&A. I mean, they do startups here and there. I mean, the one big one they went for, which actually didn't cover in the book because I ran outta time, was trying to buy Arm. And I mean, can you imagine if they were able to buy Arm for the price that they were trying to buy it? I mean, that's another $100 to $200 billion in And value Jensen would have been able to make. But I mean, that just fell apart because Jensen wasn't able to do the political stuff with regulators at the time. And Qualcomm was actually very— I have reporting where the Qualcomm CEO was out with regulators and Jensen jokes about it that by the time he got to regulators, all the regulators were talking through Qualcomm's talking points. So He did lose out on that big deal, but not that much M&A in NVIDIA's history.

Yeah. Yeah. It does feel like Jensen, he has many advantages and probably getting lots of politicians on his side is not at the top of the list of things that he's unusually good at relative to others.

Politics is very, I mean, it's the antithesis of what he likes inside his company. Like I talked about, You know, getting rid of the bureaucracy and the currying, backstabbing, internal politics. That's what politics is. When you talk to politicians, that's what you have to do. And he hates that stuff. He hates that. If anyone he finds out, anyone that's doing that internal political stuff, gaming metrics and trying to steal credit from other employees, he'll dress that down, dress down that person in public, embarrass them because That stuff is corrosive to him. And I don't know if you ever worked at large organizations, it's endemic. Like, I've worked at large companies and I see everywhere and where, you know, 30 to 50% of employee time is spent on internal politics and building alliances. It literally is like Game of Thrones where you're building alliances and trying to help each other instead of doing the right thing for the customer, which NVIDIA calls the mission is the boss. The mission is the boss. Not making your boss look good to his boss, but doing the right thing for the company and the customer. That's the mission. And that's what you should focus your energy on instead of playing political games to, you know, try to get credit so your bonus is higher in the fiscal year. I mean, that, that, that stuff drives Jensen nuts.

Yeah. And I think that's a, that's a good jumping off point for just talking about NVIDIA's internal organizational structure and their, their wild org chart where they have a CEO with 60 direct reports. So. Yeah. How did it, how did that evolve and how does that actually work?

So he said that the first 10, 15 years of Nvidia's history, they had the same political problems, internal politics that every other company had. So he wanted to create something that impeded that and would be more agile, agile, he said. And so he flattened the Nvidia corporate structure. Most, most companies are like a pyramid. Where there's someone on top of a business unit and then there's layers of people below him. He wanted to structure Nvidia that's more like a short cylinder where there aren't that many levels and that he could reach out to any employee at any time. A key component of this is this thing called top 5 emails, which you mentioned before. Most companies have status reports where a low-level employee writes a report that goes to a middle manager who sanitizes that report, sends it to his boss, his boss's boss, and by the time it gets to the CEO, there's nothing actionable in the report. It's completely devoid of any bad news, which is what the CEO needs to steer the company. So at Nvidia, there's no formal status reports. Everything is through this thing called Top 5 Email, where an employee and every executive either every week or every 2 weeks, writes the top 5 most important things that they're working on, they're seeing in the market, what competitors are doing. And what Jensen does is he samples 100 of these top 5 emails every day. So he gets a real-time view of what's going on in the market, what his employees are working on, what competitors are doing. So that helps him make strategic decisions on where to steer allocate resources almost on a real-time basis. And because he's constantly on email, like I said, he's emailing all weekend into Sunday night asking his employees, what are you doing here? What did you mean by that? He's constantly sending probably hundreds of emails a day. He has almost like a great sensor system that he sees what everyone's doing. And that culture of employees being honest to Jensen and their executives, they call intellectual honesty. It is part of the firm's culture, so it's just faster compared to the slow status reports in a typical company. Jensen has this real-time view with hundreds of data points every day, and it just works in a faster, better way for Nvidia. That's something I don't see other companies doing. I mean, most companies love to have the CEO managing six or seven direct. Direct reports where the chief operating officer is another layer doing all that stuff. Now Jensen wants his fingers and his pulse on everything all the way up and down through the company. And I think actually the other key thing is he just has the technical competence that most CEOs don't have. He's actually trading papers, AI papers, the latest academic AI papers with his engineers. He's on these discussion boards. Nvidia has all these email discussion boards for AI or Or autonomous driving, where engineers trade the latest articles and research and whatever. He's inside these discussion boards and sending the latest AI papers to his engineers. So he's passionate about these technical specifications and all the latest technology. So he's in the weeds on this stuff. One guy told me, one manager told me, Was preparing and met Jensen to talk about wireless specifications for this unimportant product. And he knew more about what's happening in the wireless specs than he did. Like, he, like, that's— it's not that important. The CEO of NVIDIA should not be, you know, look— looking at, you know, 802.11b versus n versus g. But he, he loves this stuff so much that he knew more about what was happening in the wireless standards than the product manager did. So that's, that's a special, that's a special and unique CEO that has that business genius and that technical acumen, that, that combination and just that passion to work 24/7 on all this stuff. That's what makes NVIDIA so special and different.

Yeah. So last one, because I know we're almost at time. So you've, you've been following tech companies as an analyst and as a journalist. So like, I'm sure you've seen a lot of companies come and go and You know, looks at different companies at different points in their history. Was there, is there any historical company where it feels like NVIDIA is kind of recapturing a lot of what made them special when they were at their, their peak of growth and influence? And then are there any companies that come to mind today where you look at them and you say like, they, they are pushing that same kind of set of cultural norms that made NVIDIA what it is now?

I mean, 2 companies I would compare NVIDIA to are Intel. During their glory days, Microsoft too, with Bill Gates and Andy Grove with Intel. This kind of engineering-led plus business genius is such a rare thing. And then Apple with Steve Jobs, obviously. So those are the companies I think Nvidia compares very closely to. In terms of new companies that they're doing that—

I know this is always harder. You're kind of writing a reputational put option when you say this.

I mean, nothing really comes top of mind right now. Maybe Epic with Fortnite and all that stuff. But yeah, I don't really have any large, medium-sized that are innovating at the scale of NVIDIA. Well, NVIDIA is very large now and they're still innovating and really pushing the envelope. So there aren't any obvious startups or smaller companies that have that style.

It feels, I guess that is kind of consonant with some of the things you've said in the book, because you have some quotes from Jensen where he basically says he was not a very good CEO early on and that Nvidia was not an unusually well-managed company. It was like a company with average to below average management practices that happened to have a lot of good people who basically bailed them out from avoidable mistakes. So, yeah, maybe that is the next Nvidia, is that it's a bunch of smart people and you're always asking yourself, why do they keep making these mistakes? And then you find out there's just like a little bit too much internal politics. They haven't quite figured out what the right way to structure things is. They don't know quite how to get the right information to the people who are going to act on it. And then once it clicks, once it clicks, things really start working. So yeah.

But, but I do think he was being overly negative about the early years of NVIDIA. I mean, they grew faster than any chip company in history in 12 years in. So I think that's just part of Jensen's, he does these psychological tricks. Where he's so scared of becoming complacent and being that tech company that becomes resting on their laurels, and then some disruptive startup comes up and destroys them. So he does these psychological things where the story I have in the book is they had a blowout quarter where Nvidia just blew the estimates away. And then he comes into a meeting and told his executive team, I looked at the mirror this morning and stared at the mirror and said, you suck. You suck, Jensen. But obviously they don't suck. He's just trying to bring himself down when he's afraid that he's getting too overconfident. On the flip side, he does the opposite when he becomes intimidated and things aren't going so well. He tries to trick his mind and say, how hard can it be? I mean, it turns out to be very hard when they do the new thing, but he tries to trick his mind and say, how hard can it be? And so these psychological mental tricks, I think, are part of what makes Jensen successful because he knows what he has to do mentally to pare down his overconfidence or to help him when he's feeling very intimidated or depressed. So I think part of those tricks are why Nvidia has been successful. And after this whole process, I feel like I have the AI Jensen on my shoulder. telling me what's the right thing to do in my work and personal life. And I do have this couple dozen things, these catchphrases at the end of the book. I call them Jensenisms.

I like that. Yeah, I feel like putting those on sticky notes, having them strategically placed around your home and office, it would probably drive you a little bit crazy. But it also feels like that is kind of the Nvidia story, is like you want to drive yourself just crazy enough to go slightly faster than everyone else thinks is possible. And then You don't want to be too crazy, but the optimal amount of sanity is not 100%. Yeah. Thanks a ton for coming on the podcast. Thanks a ton for writing the book. Again, it's The NVIDIA Way, available wherever books are sold, I believe is the phrase. So yeah, I highly recommend people check it out.

Thanks for having me. I really appreciate it.

network behind Moment of Zen and Cognitive Revolution. If you like the episode, please leave a review in the Apple Store. Hey everyone, Erik here. At Turpentine, we're building the first media outlet for tech people by tech people. We're the network behind the show you're listening to right now. We have a slate of hit shows across a range of topics and industries, from our AI and investing cluster of podcasts, to shows that drive the conversation in tech with the most interesting thinkers, founders, investors, and influencers, like Econ 102 with Noah Smith. We're launching new shows every week, and we're looking for industry-leading sponsors. If you think that might be you and your company, email me at erik@turpentine.co. That's erik@turpentine.co, and let's partner together.