What Everyone Is Getting Wrong About Sequoia's $600B AI Gap Argument

Today on the AI Daily Brief, we're exploring whether AI is in the mother of all bubbles. And before that, in the headlines, OpenAI's China ban goes into effect. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show. Notes. Welcome back to the AI Daily Brief Headlines edition. All the daily AI headlines you need in around five minutes. One of the big themes running through the AI space is the geopolitical battle and tension between the US and China. This is not new, nor is it exclusive to AI, but it has certainly been heightened because of the rise of AI. One of the recent developments is that last month OpenAI announced that they would be blocking Chinese users from their products. Now, of course, technically ChatGPT was already blocked in China by the government firewall, but before this new block from OpenAI, developers could theoretically use VPNs to get around the firewall and use OpenAI's API APIs to fine tune their own AI applications or benchmark their own research. Now, however, the block is coming from both sides at once, said an OpenAI spokesperson last month. We're taking additional steps to block API traffic from regions where we do not support access to OpenAI services. Although it was announced last month, the block went into effect today, July 9, according to leaders in the community. This move from OpenAI, which by the way, was not explained at all, has, quote, caused significant concern within China's AI community. However, for domestic Chinese companies, it also creates a new opportunity. The way the Guardian puts it is that companies like SenseTime and Baidu, quote, scrambling to hoover up OpenAI's rejected users by way of example of how that competition is playing out. After the decision was announced last month, Baidu offered 50 million free tokens for its Ernie 3.5 model. Zhipu AI offered 150 million free tokens, Tencent Cloud was giving away 100 million free tokens and SenseTime also giving away 50 million free tokens, writes the Guardian. One consequence of OpenAI's decision may be that it accelerates the development of Chinese AI companies, which are in tight competition with their US rivals as well as each other. While US companies such as OpenAI have been at the cutting edge of generative AI, Chinese companies have been engaged in a price war that some analysts have speculated may harm their profit margins and their ability to innovate, said NYU Professor Vincent Ma. OpenAI's departure is a short term shock to the China market, but it may provide a long term opportunity for Chinese domestic LLM models to be put to the real test. We also got news that in another workaround, OpenAI's China ban would not apply to Microsoft's Azure China. Basically, Chinese companies can still get access to OpenAI's APIs as long as they are buying them through Microsoft's Azure cloud service, which operates as a joint venture with a local company, 21Vianet. The information went out and looked and found three Azure China customers that confirmed they have access to OpenAI's models, writes the information. The divergence between Microsoft and OpenAI on this issue reflects broad differences between how each company deals with China. OpenAI may feel a need to avoid any suggestion that its technology is helping Chinese companies stay competitive with American ones, given Washington's concerns about China advances in the area, the information continues. The contradiction between the policies of OpenAI and Microsoft became particularly apparent late last month. A day after OpenAI notified Chinese developers about its crackdown on access to ITS Services, Microsoft China's official WeChat account, published a post in Chinese encouraging developers to migrate to Azure OpenAI. Now Microsoft has some of its own challenges when it comes to DC and AI. We recently covered how their deal with G42, which at the time frankly had seemed like it was set up by the government, was increasingly under scrutiny given that UAE based companies ties to Chinese companies as well. Interestingly, a new survey helps dramatize the stakes of all of this. In a survey of 1600 decision makers in industries worldwide by SAS and Coleman Parks research, 83% of Chinese respondents said they used generative AI, compared to, for example, 65% in the United States and and 54% globally. Last week, a report by the United Nations World Intellectual Property Organization showed that between 2014 and 2023, 6,276 patents had been filed in the US around generative AI, as compared to 38,000 filed in China. Then again, not all of the ways in which China is using generative AI are things that Americans are particularly keen on. For example, the SAS report points out that China leads the world in continuous automated monitoring, I. E. Professional surveillance. Couple more quick stories before we get out of here. One more on OpenAI In a very different dimension, OpenAI has partnered with Arianna Huffington's Thrive to create an AI health coach called Thrive AI Health. In a Time magazine op ed, OpenAI CEO Sam Altman and Thrive leader Arianna Huffington said that the health coach will be trained on the best peer reviewed science alongside the quote, personal biometric lab and other medical data. You've chosen to share with it. Now this is an area where there is a huge amount of experimentation and development when it comes to the use of AI, and I think the interesting thing here is just OpenAI choosing to explicitly move into this space through this partnership. Lastly, from the annals of tough startups, the latest news out of Humane is that two of its executives have left the company to found an AI fact checking startup. Former strategic partnerships lead Brooke Hartley Moy and head of Product Engineering Ken Cocienda have started a company called Infactory, which is described by TechCrunch as a kind of fact checking search engine. So far we only have the information that's been reported, but it's supposed to be basically a more sophisticated search system that knows when and when not to use AI and I think underpins just how much competition there is still in the quote unquote AI search space. Still, for most people, the big story is Humane losing execs and I wonder if that portends more trouble for the company. For now though, that is going to do it for the headlines. Next up, the main episode. Today's episode is brought to you by Superintelligent, the platform for fun, fast AI learning. Super has a ton of new things going on. We recently announced our partnership with Spotify, through which users of that app can now access super intelligent content directly from their mobile apps. We've also just launched the AI Learning feed. In addition to seeing the tutorials that we are dropping, there are polls, news items with related lessons and a chance for people to show off the projects and use cases that are making AI come alive for them. We've also just kicked off the Super Summer Challenge where each week we'll share a new challenge that you can use to discover new AI tools and use cases. Go to BeSuper AI and use code SUPERFUND for 50% off your first two months. That's BeSuper AI. Today's episode is brought to you by Venice. The leading AI companies store your entire conversation history and attach it to your identity forever. Every question you ask, every answer you receive, every image you generate, every thought you share with the machine, it's all being spied on. If you trust all the company's hackers and NSA board members that will ever have access to your AI conversations, then rejoice for you are well served. For the rest of us, Venice is an alternative. Venice is a powerful AI app for text, image and co generation that respects you as a sovereign individual and believes privacy and free speech are not only human rights but are necessary for civilizational advancement Private, permissionless and uncensored. You can try it for free without an account at Venice AI welcome back to the AI Daily Brief. Today we're starting to dig into a question that is something of an emerging narrative, like which is the idea that generative AI is in a bubble. Now, one of the biggest progenitors of this thesis is a recent piece by Sequoia called AI $600 billion question. However, before we get into that piece itself, let's look at some recent data that shows just how significant AI is when it comes to early stage funding, Reuters reports last week AI deals lift US venture capital funding to highest level in two years According to PitchBook, in the second quarter of 2024 US VC funding was up to 55.6 billion. That is a 47% jump from the 37.8 billion that US companies raised in the first quarter, but still nowhere near the record high of 97.5 billion from the fourth quarter 2021. Unsurprising. If you've been paying attention, a huge part of that went to AI companies. Some of that was in big individual rounds like the 6 billion that went into Elon's XAI and the 1.1 billion that went into Coreweave. Axios also points out that AI accounted for more than 40% of new US unicorns, I.e. companies worth $1 billion or more, and that AI VC deals accounted for over 60% of the total increase in venture backed valuation. In the first half of this year. 13 new AI unicorns were minted in the US. What's more, the valuations are higher. AI valuations were double digit percentages higher than categories like FinTech and SaaS. Now of course there are questions around where this is all headed, reuters writes. Despite the increase in deal activity, exits remain challenging. Small deals generated about 23.6 billion in exit value in the second quarter, down from 37.8 billion in the first quarter. They also pointed out that this lack of exits may be impacting venture fundraising itself. Pitchbook pointed out that only 37.4 billion had been committed to VC firms in the first half of the year and that within that total it was dominated by big firms like Andreessen Horowitz, who closed funds worth more than 7 billion. A big question in the VC side of this market is whether an M and A market for AI startups will start to pick up steam. Discussing companies like Nvidia and Databricks, VC firm Section32 CEO Andrew Harrison said they put down venture dollars first and watch how it evolved and started to shake out. Now I think they're more serious about which pieces of the puzzle they want to own as they're starting to see the emerging winners. In other words, some people are betting that all of this corporate venture capital that's been going into AI companies is going to start to resolve into actual acquisitions as well. Certainly when it comes to AI, VC firms are thinking about how to differentiate the Information Today reported that Andreessen Horowitz has quietly built a stash of more than 20,000 GPUs in an attempt to win premium AI deals. By way of comparison, that's around the number of GPUs that Xai used to train Grok, the information writes. The program highlights Andreessen Horowitz's aggressive moves into generative AI in the last two years. It has likely cost the firm hundreds of millions of dollars based on the current cost of specialized AI trips. But they also wrote that it's not clear whether A16Z has purchased the chips or is renting them. A16Z is of course, not the first firm to do this. Nat Friedman and Daniel Gross, who have been some of the most active investors in the AI space, last year acquired around 2,500 H1 hundreds worth around $100 million to again provide access to that compute to their startups. At the same time, times may be changing when it comes to GPU again, the Information writes. Signs the chip shortage has started to lessen have already prompted at least one such investor to change tac. Sarah Guo's early stage venture firm Conviction last year paid a cloud provider for access to GPU servers and made those servers available to startups at the cost the firm paid. Then, as GPUs became more available, Conviction reduced its orders and put some of its servers onto chip marketplaces to sell. They also point out that at least one firm, in this case Sequoia Capital, have called a top to the supply crunch, pointing to a blog post from June where Sequoia partner David Kahn called late 2023 the peak of the GPU supply shortage. And this brings us to this essay, which has been widely referenced as evidence that AI is in a bubble. The essay was called AI's $600 billion question. The AI bubble is reaching a tipping point, and navigating what comes next will be essential. The piece at Core is a gap between what Khan calls the revenue expectations implied by the AI infrastructure buildout and actual revenue growth in the AI ecosystem. In September 2023, Khan identified that gap as around 125 billion, but now sees the gap at more like $500 billion. In terms of how he calculates this, Khan writes, all you have to do is take Nvidia's run rate revenue forecast and multiply it by 2x to reflect the total cost of AI data centers. GPUs are half of the total cost of ownership. The other half includes energy, buildings, backup generators, etc. Then you multiply by 2x again to reflect a 50% gross margin for the end user of the GPU, eg, the startup or business buying AI compute from Azure or aws. Who needs to make money as well? So what, Khan asks, has changed since September 2023? First, he says the supply shortage has subsided. That's where he called 2023 the peak of the GPU supply shortage. Second, he says GPU stockpiles are growing. In Q4, for example, about half of data center revenue came from the large cloud providers. Microsoft alone represented 22% of Nvidia's Q4 revenue. 3 Kahn writes OpenAI still has the lion's share of AI revenue, saying that while OpenAI is up to $3.4 billion in revenue, the gap between them and everyone else quote continues to loom large. 4 and I think this is the most significant. He writes, The $125 billion hole is now a $500 billion hole. In the last analysis, he writes, I generously assume that each of Google, Microsoft, Apple and Meta will be able to generate 10 billion annually from new AI related revenue. I also assumed 5 billion in new AI revenue for each of Oracle, ByteDance, Alibaba, Tencent X and Tesla. Even if this remains true, and we add a few more companies to the list, the $125 billion hole is now going to become a $500 billion hole. So at this point it's worth asking what Khan and Sequoia are really talking about here. And I think this is extremely important because if you just look at, for example, X or the people showing up on the morning business talk shows, you would think that storied venture capital firm Sequoia is calling the entire AI sector a bubble which is bound to burst and wipe out tons of value, proving that AI is just a hype train. First of all, simply by the actual words of the piece, that's not the conclusion that Khan comes to. He writes, a huge amount of economic value is going to be created by AI. Company builders focused on delivering value to end users will be rewarded handsomely. We are living through what has the potential to be a generation defining technology Wave companies like Nvidia deserve enormous credit for the role they've played in enabling this transition and are likely to play a critical role in the ecosystem for a long time to come. So, hardly a full throated argument that AI is just a hype train. What he does call a delusion is the delusion that says we're all going to get rich quick because AGI is coming tomorrow and we all need to stockpile the only valuable resource, which is GPUs. So when we're looking at this piece, what Khan and Sequoia are actually talking about is not the value of AI in general. It's not about whether AI startups can find or create big markets. It's not about whether enterprises deploying AI are making a good investment. It is simply and specifically about the capital expenditures of the largest companies, the hyperscalers that are training their own models, building out data infrastructure and hoping that at some point this actually turns into real revenue value. In other words, you could redefine this question, this $600 billion question, to be about whether the stock market specifically is pricing this correctly. Are investors, in other words, pricing the AI buildout and the race to AGI in an appropriate way? This, I think, is an interesting and complex question. On the one hand, this gap between infrastructure build out and realized revenue becomes a lot more relevant, especially in the short term, which is of course, the horizon that Wall street investors have. The factors that Kahn identifies, such as GPU stockpiles growing, the depreciation of capital assets, things like that, all will have an impact on that question. Of course, on the flip side, there remains the big open question or X factor of how valuable AGI will actually be, how transformative. In other words, these technologies are at maturation, and of course, how long it takes to get to that state. These are all reasonable things to debate. And there is likely going to be a lot of money made, not only betting on the future, but but going short on how fast that future gets here. Ultimately though, that question is not about AI, not really @ least. It's a question about market pricing. I've often said that one of the things that makes this moment so strange is that usually when you have a technology paradigm shift, the stage that we're in happens almost entirely in the private markets. There isn't the same sort of role for the big companies, the big tech players, as there is when it comes to AI. These dynamics are simply unlike anything we've seen before. And because of that, there is a much wider diversity of opinions around how it plays out and how it should be valued in the short term. Unfortunately, our media infrastructure is not capable of dealing with that sort of nuance and instead just focuses on the questions of whether everything's going to moon or whether everything is a big bubble or whether it's both and when the bubble bursts and when the moon comes. Certainly there is interesting evidence that the leading companies do believe that something about the dynamics now are unlikely to last forever. In the middle of June, the Information wrote a piece about exactly this. Nvidia's Jensen Huang is on top of the world. So why is he worried? The piece is about how Jensen is pushing Nvidia into software and cloud services and trying to diversify its revenue away from just the data centers, writes the Information. Huang told colleagues he was worried cloud server providers such as AWS and Microsoft, which collectively have been buying about half of Nvidia's AI server chips in recent quarters, weren't moving fast enough to set up new data centers and power sources to accommodate the chips they had ordered. Huang and his colleagues have also focused on countering the next threat to the business the likelihood that demand for Nvidia's chips will eventually slow down. Now, Nvidia's answers to that, at least for the purposes of this podcast, matter less than the fact that that's what they're thinking about. On top of that, enterprises are also getting more sophisticated about how to appropriately integrate AI and what does and doesn't matter in terms of what they're buying, stephanie Palazzolo, once again from the Information, recently wrote. Businesses want slower AI models and that might hurt Nvidia, she writes. There's surely more money in business focused AI services than in the consumer market, at least in the near term. By the same token, though, businesses have become much more focused on the cost effectiveness and returns on AI than on finding the fastest, most advanced model. One sign of that is the rise of batch processing. Today, popular consumer AI products like ChatGPT and Perplexity provide users with near instantaneous responses, otherwise known as real time inference. However, businesses don't always need immediate responses and are often willing to wait hours, days or even weeks for responses, as long as they don't have to shell out as much money. Founders of cloud and inference providers have told me there's growing pressure from business customers for this kind of flexibility, otherwise known as batch processing. To me, this is a complete counterweight to this idea that we're all just in a bubble. It shows a sophistication on the part of enterprises to figure out which types of use cases they need AI for and on what timescale. The point of all of this is that ultimately, if you want an informed view of whether AI is in a bubble or not, you must get beyond the simplicity of that question. The answer to whether there is a Wall street market bubble in terms of how big tech is priced is fundamentally different to whether there is a bubble in hype surrounding enterprise use cases. And that of course, is fundamentally different than how AI is showing up in individual consumers lives. It is obviously the unspoken mission of the AI Daily Brief to help you parse through this stuff, and I appreciate you hanging out as we do. For now though, that is going to do it for today's AI Daily Brief. Until next time.