Vinod Khosla on AI’s Credit Risks, High Costs, and Energy Demands

Vinod, welcome to TITV. It's great to have you here.

Well, it's always exciting to be here.

So before we get started, I just wanna set the stage a little bit because our subscribers at The Information know your career very well. They will know that you don't consider yourself as much of a venture capitalist as you do a venture assistant. They will know that you believe that AI is going to replace 80% of jobs in the long run. And they will also know that you are a steadfast optimist. And one of the guiding mantras in your career has been that skeptics never did the impossible. And so with all that in mind, I want to zoom in on that third point because it feels like right now, even for people who are very bullish on AI, the technology, there is enough reason to be at least somewhat skeptical on AI, the business. And so where I want to start the conversation is, Vinod, is there anything that you are skeptical about right now?

Well, to start with, I think if you're going to do something significant, you're going to have to be an optimist, but not only just an optimist, a knowledgeable optimist. There's too many skeptics who live in the past and will extrapolate the past to invent the future. That's a bad thing to do. Inventing the future you want is the right way, and optimism is key to that. Having said that, I think the one thing I don't know, and there's things I can reasonably project and things I can be uncertain about. The thing I'm uncertain about is timing of various breakthroughs that we expect to see in the next 2 years, 5 years, 10 years. Almost certainly it's easier to predict AI in 2035 and past probably 2030 than it is to predict it today and what will happen next year. Right. So I'm not skeptical, but I'm less certain about the pace of development of certain capabilities that are essential to expanding use of AI. Right.

Well, let's talk about the, the, the current state of play. I want to get a little bit more granular here for a moment. One of the topics that has been talked about a lot in our coverage are the margins that a lot of these companies have on their AI products, whether it's cloud services or even AI application companies. And one of the challenges that we've highlighted in our coverage is that margins are slim right now and companies are finding ways to expand them. How do you think this goes in the long run?

I think the way to look at margins is when AI is a substantial business. Yes, it's large business, tens of billions of dollars, but it's not hundreds of billions of dollars. And so the question to ask is where will margins stabilize? Uh, I think some of it depends on R&D breakthroughs by the various companies and Knowing who gets what breakthrough, which is, as I said, uncertain with respect to who and timing. I'm pretty optimistic about what OpenAI is doing. I do think if you provide value and you have differentiated models, you will have good margins. Look, at the pace at which pricing is declining both in supplying, uh, inferencing to the, uh, AI companies and the price at which things are sold. Uh, it's hard to predict the dynamics, but will settle down. Good returns on capital, uh, invested. So I'd be surprised if we don't have healthy increasing margins well into the, into the early 2030s.

And, and, and which of the two levers do you think is more likely to get pulled on most? The idea that will costs go down for these chips that are very expensive right now? Or on the flip side, we had one of your founders on the show, Amjad Masad from Replit. He talked about this idea that the models might not get cheaper, but rather he's looking to actually increase his prices to widen his margin. So if you think about what people are willing to pay, and then the cost of the chips. Which one of those two levers do you think is most likely to widen margin in the next couple years?

So there's a number of ways in which cost of supplying inference will go down. One, chips will become more economical in per inference. I do think the algorithms will get much better, which means the software will get better in the amount of compute needed per inference. So there's two vectors. You can make the algorithms 10x more, more efficient over time. I think that will happen. The cost of chips will go down for a given number of inferences. So both those are vectors for cost reduction. The question of pricing on the input side, what customers will pay, will be a function of value. If Amjad, and I love that company and what Amjad is doing, if he keeps adding more and more value, he will be able to charge more. So that's a question of value addition. And that's where there's lots of headroom. You know, when you're taking something that costs, say, a professional, an accounting professional, for example, or, um, or a design product designer, and you're paying them $100 to $300 an hour, and your cost is $1 to $3 an hour, you have lots of pricing room if you can complete— provide more complete solutions. So I do think price per hour of worker time equivalent will increase pretty dramatically also, as will, uh, as will the decline in cost per— so how do you square this model? This business will be healthy starting, let's say, 2030 and beyond, when scaled data centers will be in operation.

How do you square that with this matter of enterprise software companies, for example? Buyers of enterprise software, I'm talking about businesses in general, they are struggling to find the value right now, at least on an ROI basis for much of this AI software. I mean, the reason I'm asking the question is because we're talking about raising prices. They're not seeing the value right now. They're very slow to adopt this AI software. So how do you square those two realities right now?

So you have to look, at a couple of factors. One, where is value being provided? Great. In software development, it's absolutely great value. So companies like Replit and Cognition, which we are both investors in, Vinod Khosla included, all growing very, very rapidly, uh, because they're adding real value. Uh, so there are functions in which the product isn't complete or mature. And so you need almost so much handholding, the economics break down. I would say another factor that most people haven't considered, most enterprises who are executing AI are doing it with their people who are not qualified to execute. It's like saying, hey, we have a race car and Joe Blow can go drive it. And he's not going to get most of that race car.

So they need to hire different people.

I think they need to take a very different approach. They take an online IT software person, say, build an agent for me and hope it works great. It's not the way it's working. So generally, if you take a company like Distill, which is in our portfolio, if they execute a project for a large Fortune 500 company, it goes swimmingly well. If in-house people in the same company execute it, it goes very poorly. So, uh, in each of these, even the in-house people will get better over time. So the third or fourth generation of execution will do better because their people will get trained, but they're really not qualified to operate in this area. While the AI-native companies are able to do, get real value. And they're even able to pick the projects that will be valuable and the projects that are more experimental.

I want to pivot to talking a little bit about some of these circular deals that we've seen happening in the AI sector. I mean, NVIDIA, for instance, has been at the center of some of these circular financing arrangements. Do these concern you at all?

Well, they don't. They do and they don't. It's hard to tell what the details are behind each of these debts. If NVIDIA is financing customers to buy their chips, that could be perfectly reasonable. The question So, there are many, many industries. Look, General Motors finances its cars too, right? When a consumer buys it, it's just a regularized business. The question is hidden in the contracts that are mostly not publicly available. Who's taking on what part of the risk? Is it an enterprise? Are you saying a customer? Your financing is viable or not? Are you saying the risk is the customers or their customers if they've done a contract with somebody else to buy X million dollars of inferencing? So where's the risk hidden? Well, and, and contracts is the key question before one can apply. And frankly, most of those risks and who takes what risk is in the fine print and not visible to almost anybody outside.

I take your point on the fine print. I guess what I'm sort of trying to assess here is the systemic risk in the circular financing at large. And of course, I take your point about, for example, General Motors financing the purchase of a car, but in a world where you have a chip company financing the— investing in OpenAI, OpenAI buying cloud compute from Oracle, Oracle buying chips from NVIDIA. I mean, that is a sort of circular loop that I think a lot of people have sort of raised flags about. Does that whole cycle not raise any flags for you or cause any concern?

Well, I would say I don't care. I don't care for the following reason. If NVIDIA is taking a bad credit risk, that's their problem. Right. But if NVIDIA loses $50 or $100 billion, does it kill the company? Probably not. Uh, and I suspect Jensen's pretty smart about what credit risks he's taking with which customer. Is Oracle taking a larger risk? Possibly. It depends on the details of the contract between NVIDIA and Oracle and Oracle and OpenAI or other people buying their, their cloud service. So you have to think of it as traditional business and say, where does the risk lie? If Oracle goes under, for example, because they, they took the risk of $100 billion spend and didn't get that back, then it's their problem. If Oracle disappears from the scene, do I care? No. Uh, I hope they don't. I think the ecosystem will be healthier, but people are selectively taking risk. Corvee is taking a lot of risks with the money that belongs to certain lenders to Corvee. Do the lenders have risk? I don't know. Does Corvee have risk? And Oracle's doing the same thing. It's taking risk, but I don't know the details of these contracts.

So, But broadly speaking, the level of risk that this has become sort of ubiquitous throughout this AI ecosystem, that doesn't concern you at all?

I would say the fundamental notion of will there be more demand for API inference calls doesn't concern me at all.

Right.

Will there be, the fundamental is how many inferencing calls we'll see in 2030? And 2035. That generally doesn't concern me because I believe AI will add a lot of value. Look, the US economy is $15 trillion of labor alone. Just labor costs in the US economy, $15 trillion. If you could replace $5 trillion of that, there's plenty of room for inferencing to be paid for. If you can do that. Uh, so again, I said the fundamental is, is there demand for AI inferencing the next 5 years, the next 10 years? I'm not worried about that.

Right. Who do you think has—

who's going to find how clever a contract and who takes on risk if demand is slower or faster to emerge? No, that's for individual companies. If they did a bad deal, they'll go under. If they didn't, great. If everybody does great, which is possible, that AI just grows so fast that nobody has any risk, great. But I'm not responsible for a lender financing a data center. If they fail, their problem, not mine.

Right.

I care about the innovation ecosystem that drives more API calls in AI.

Who do you think has the best chance of challenging NVIDIA?

Well, obviously AMD is trying things, uh, Arm is trying things, Broadcom is trying things. NVIDIA is in an enviable position, uh, because they have so many different things they can do in parallel because the cash flow they have, right? Now, do I know all of Jensen's plans inside and how many different things he's Klein? No. In fact, nobody outside really knows when he's going to announce what. My bet is he has a pretty precise roadmap to 2030 and beyond. So hard to say, but—

So there's no one company that you are really sort of putting your eggs in here in terms of, I think this company is closest on NVIDIA's tail right now.

Well, AMD is doing pretty well by signing, looking at their deals. Broadcom's doing pretty well, but are they gonna grab majority share and be larger than NVIDIA? I wouldn't expect that today. Can they take reasonable share, especially at slightly lower margins? Yes. Hmm.

Now, what about all these chip startups that are popping up?

Well, I haven't— I've seen a lot of specialized chip startups that do one thing. You can run your whole model locally in your shop. Well, that's, that's a market. It's not as large as the data center inferencing market. So there's many sub-markets that the chip startups can do okay in. But I haven't seen the chip startup that could completely blow everybody away. Now, if you have a sudden breakthrough in photonic chips that can do multiply-accumulate inferencing functions, and cut the power consumption by 70% for inferencing. That's entirely possible, even likely, sometime in the next 5 years. And I hope some of those show up because it'll change the power equation, how much power we need for AI. And frankly, most of the data center investment can be repurposed with a new kind of chip. That gets slotted in.

I want to tell you—

Photonics is actually pretty promising. I suspect digital semiconductor chips are going to be hard to beat NVIDIA at in a massive way. In specialized segments of the market, you can beat them. Right. But generally, you'd have to have a radical breakthrough in technology. Photonics is one of them. There's a few others. Uh, but I see the most promising candidate for an alternative to NVIDIA becoming from photonics, if one can scale it. And photonics typically is hard to scale. Now, we were builders of one, uh, one of our portfolio companies a long time ago, Infineon, was built the first photonic chips ever. So I think we understand that space a little bit, but we'll see what comes along. Right.

I want to close by talking about the energy side of, of the AI equation. You, of course, have been in energy for a long time. You've made big bets on fusion, among many other technologies. And the question that I want to help get your perspective on is this idea that a lot of these energy bets are going to still take years to scale. Meanwhile, the energy demands that AI will need to sort of satisfy. I mean, those demands are right now. And so, help us reconcile this idea that we need power now and it's going to still take several years for many of these new energy technologies to scale.

So, the simplest way, very, very short term to address electricity demand in the country is pricing. Vinod Khosla: Look, it's— this is why the economics of marketplaces work. Prices will go up some as we consume more pricing for data centers. Data centers themselves can get to be aggressive sources of power management. You can consume at certain times of the day for training runs and other times of the day for inferencing. You can dial up the performance or dial down the performance. So data center input of electricity itself is a variable. That'll probably be part of electricity trading. There's some good startups in that area. I think there's a short-term solution, which I think is Superhot Geothermal. I think we can get to gigawatt-scale If you imagine a couple of extra gigawatts of demand emerging every year, some of it will be met through pricing energy appropriately. Some of it will be met through shorter-term projects. Geothermal is one that's much shorter-term than, say, fusion. Hmm. Fission's a possibility, but I think fission to me will take the longest. Even compared to fusion. But we'll have an array of factors. We'll have more natural gas turbines coming on. We do have companies where you can start with natural gas and switch to hydrogen whenever the economics warrant it. So, there's a number of ways to adjust, but it is a non-trivial problem.

And, and, and so all these data centers that companies like OpenAI are springing up very quickly, I mean, The simple question I wanted to get your take on is, do we have enough power for these facilities?

Well, first thing to keep in mind, it takes a couple of years to build a data center. You're building a—

But many of them are coming up much faster than that, right?

There's a few hacks, but fundamentally, if you're trying to add a gigawatt of data center, which is about $30 billion of spend, It'll take a couple of years. I don't think that's a 6-month or 12-month or even an 18-month project. Then there's demand-based electricity consumption as a tool. And then, yeah, I think things like geothermal and other technologies will come along. Some of them will be natural gas-fired. I hope there's no more coal facilities. You know, Mainspring installs capacity for data centers that can switch seamlessly from natural gas to hydrogen when you want to go clean. So, you can decide how much you want to pay for power and what carbon reduction you want and increase the carbon reduction over time. So, that's one solution. All I'm saying is there's an array of tools. Not non-trivial. This will be a Serious issue and policy is trying to address it, but I do think there's many solutions.

Last question for you. You know, you wrote this op-ed for us a couple months ago in The Information. It was called, well, it was about the bonkers valuations in AI right now. And one of the things that you mentioned in the piece was that you think that venture capital as an asset class is likely to shrink over the coming 10 years. And I wanted to ask you what the repercussions of that are in your mind. The obvious one that I thought was perhaps startups might actually get better because there's less capital to go around and the better ones will get picked. Is that the main repercussion of this or what are the other repercussions that you see happening?

Let me suggest, you know, I think that op-ed was misinterpreted a little bit. What I said was, AI valuations in general are bonkers. For the best companies, they're not bonkers. And if, as a venture capitalist, you have access to those 2, 3, 4% of the startups, there'll be huge wins. You will do well. You will have great returns. The people who are plying money in, Uh, without having special access to these opportunities for whatever reason, uh, will suffer in venture capital as a class. I think broadly for funds raised in '24 and '25 will have decreasing returns, returns will lower because they're not getting access to good deals early. They're paying higher prices much later. Some of the robotics valuations are getting bonkers. I would venture to guess 95% of those startups will lose money.

So, so the take home here is we need special access to deals essentially to be successful.

It's more than that. I think I like to say most AI startups will lose money, but more money will be made than lost. That means it'll be highly asymmetric. 2-3% of the startups will account for 85-90% of the valuation by 2035 of market cap of companies. So that asymmetry, which has generally been true in venture capital, but we will be significantly more asymmetric in AI because of this valuation wave, I think is the reason we will see average returns decline and the best returns for the top firms will do okay, will do well, in fact, because AI is such a large opportunity.

Right. Great. Well, I think that's a good place to end it. Vinod, thank you so much for being here. That is Vinod Khosla here on TITV. We will have you on the show again very soon.