The State of the AI Debate

The next era of the AI debate is officially upon us. On the one hand, you have President Trump calling for a new AI force, as well as suggesting his own rebrands of the entire industry. On the other, you have regulators and politicians all around the country proposing AI kill switches and new bills and changes to data center policy. And behind all of that, you have the lurking question of China. If China isn't on board with any sort of pause or pacing the frontier, do those actions even matter? This week, Chinese President Xi Jinping is in Washington, and already the trade delegations have been discussing AI. So will we actually see progress on some of these questions, or will the state of the AI debate remain firmly divided between these two very different countries? The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. All right, friends, quick announcements before we dive in. First of all, thank you to today's sponsors, KPMG, Blitzy, Section, and HyperAgent. To get an ad-free version of the show, go to patreon.com/AIDailyBrief, or you can subscribe on Apple Podcasts. And to learn more about sponsoring the show, send us a note at sponsors@aidailybrief.ai. At aidailybrief.ai, you can also find links to the archive of shows, to each individual episode's companion page, which has the key numbers, the key facts, the key quotes, And you can also find links to our daily companion newsletter, as well as special programs like the Multiplayer AI Sprint, which is live right now. We kick off today with Anthropic punting their planned blockbuster IPO into November, although insiders say the delay has nothing to do with AI safety. The Wall Street Journal reports that Anthropic will not, in fact, go public this month or next as originally planned. However, sources said the decision to delay was made before CEO Dario Amodei called for an AI slowdown. Instead, Anthropic's advisors have said the delay would allow them to share 3rd quarter financials, which are expected to be strong. The information confirmed the reporting, adding that Anthropic has not begun meeting with Wall Street analysts and institutional investors. These roadshows usually take place around a month before the IPO. However, some potential investors have been meeting with Anthropic as part of so-called testing the waters meetings. Leaking a few financial details, sources said that Anthropic has gone from spending $2.30 for every dollar in revenue in Q2 of last year to being slightly profitable on that basis in Q2 of this year. That would line up with recent reporting from the Financial Times, which said Anthropic claimed they had positive operating income after stripping out stock-based comp. That report repeated a claim that gross margins were above 80% if you strip out revenue-sharing deals and the cost of model training. Industry critics were perhaps unsurprisingly extremely negative on this framing, suggesting that stripping away major costs to get to profitability on paper is not a great indication of financial health. This is one potential issue with the Anthropic IPO, as the disclosure documents will give investors their first look at unadjusted numbers. The Information wrote that some investors also walked away from meetings with the impression that profitability would be short-lived. The company has signed a ton of new data center deals that will start to add structural costs over the coming quarters. Kevin Xu of Interconnected Capital suggested the market could have a difficult time signing a blank check, commenting, Anthropic going public means it is elevating itself to the entire financial market, making it more palatable for them to get different options of financing. But the public market community still wants to trust the fact that Anthropic can fund its own capital expenditure, and that debt is a bridge to that, not that the borrowing has no end in sight. That's the problem the entire AI complex has right now. The New York Times reported that Anthropic said they're on track to reach $100 billion in annualized revenue by the end of the year, up from $65 billion at the end of July. What's more, beyond the numbers, some on Wall Street are beginning to view AI safety risk as a market risk. Former NASDAQ executive turned consultant Karen Snow said that following the calls for a slowdown, the Anthropic IPO, quote, went from being a no-brainer to not being a no-brainer. She added, the markets can be finicky and sentiment has shifted a bit on AI. The issue is there's a bit of a gray cloud over AI right now. To that end, Anthropic could be delaying their IPO in order to get a new model out the door and try to shift the narrative from safety to progress. AI leakers are starting to report sightings of new Sonnet, Opus, and Fable models in stealth testing, meaning we could see a complete model overhaul in the coming weeks. It seems Anthropic are still on track for an IPO by the end of the year. Maybe the even more interesting discussion around Anthropic, however, is that the company has reportedly set up a wet lab in the Bay Area to begin biological experiments. Reuters reports that Anthropic has quietly established their own biology lab to conduct in-house research. Anthropic's head of life sciences, Eric Cadoree Abrams, told Reuters, We believe that to do biology, the final test is still, and will be for a while, in real lab work. We absolutely are doing that today. He said that the lab operates similarly to other biotech labs in the Bay Area, and while Anthropic didn't divulge exactly what they're working on, they did note the focus was on fundamental biology rather than drug discovery. Now, any biology lab that conducts physical experiments with live tissue and blood samples is considered a wet lab. However, we don't know the lab's biosafety rating, which determines whether it's able to work on risky experiments. Still, many latched onto the obvious safety concerns, with Flexport CEO Ryan Peterson asking, If Anthropic is worried about rogue AI killing everyone, why did they give it control of a robotic biology lab? Investor Chamath Palihapitiya wrote, the group behind such hits as We're All Going to Die and Regulate Me Now are building a wet lab in San Francisco. I do not recommend this. There is another really interesting question around what this means for Anthropic's business model and the business models of the labs more generally. The reporting noted that Anthropic has been careful to avoid the appearance of competing with partners in the pharmaceutical industry like Novo Nordisk. However, investor Nick Carter wrote, this is a very cynical interpretation, but could it be that the fearmongering of recent weeks is a way to gently introduce their investor base to the concept that Anthropic will be internalizing all major breakthroughs rather than letting the unwashed masses share in the spoils? Selling tokens is probably a better business model as opposed to relying on very lumpy biotech and other scientific discoveries, but if distillation really is an unacceptable margin compressor, Anthropic is incentivized to keep the frontier private and capitalize themselves on new IP rather than letting their clients have them. And, instead of outright admitting their token merchant business model is unsustainable, Anthropic could announce they are pacing the frontier and hold the general public 6 months or a year plus behind the true frontier, available only to their own internal researchers. Of course, this would require the cooperation of their archrival OpenAI, so Anthropic didn't appear to be too behind— all the better if they created a legal cartel and voluntarily agreed to slow the public frontier. They'd have to ask the government for an antitrust waiver, though. Really activates the almonds, doesn't it? Nic concludes, this is speculation. Perhaps the fear really is genuine and this all happens to be an extremely convenient state of affairs. I think there are 2 very different things going on in Nic's analysis that are actually separable. The first is whether there is a concern on the part of the labs that token prices get competed down, and in the long term the business model of selling that basic commodity doesn't look as good as it does today, and that it may be beneficial instead to take advantage of having early access to the state-of-the-art, and not just selling tokens to people to do things but doing those things themselves. That is, I think, separable from whether pacing the frontier is a vehicle for that strategy, and even if one doesn't take that cynical view, the first part, about whether this signals a sort of business model hedge among the labs, is something interesting to consider. And for what it's worth, there are plenty of non-cynics out there who see Anthropic as a lab that has clearly been excited about the possibility of the medical solutions use cases of AI, simply taking an obvious step in that direction. Now moving back over into markets, we got some first indicators of a potential data center debt crunch. The Information reports that bonds tied to a data center leased by trading firm Jane Street have sold off. The bonds were issued in August at an 8.9% interest rate, But they're now trading at 11.3%, a large increase that can't be explained by the Fed's rate hike. Now, there are a few important caveats to fully understand the implications of what's happening in these funding markets. Firstly, this doesn't change what Jane Street's data center partner is paying on the debt, which was issued at fixed rates. But it does imply that investors will demand a much higher rate for the next tranche of funding. This is also specifically about deterioration in the junk bond market rather than investment-grade debt. Jane Street carries a BB rating, and S&P Global applied the same credit rating to the joint venture building their data center. Essentially, this is just a sign that data center funding is slowing down at the margins. Another big story over recent days was that Meta has tapped the junk bond market for the first time in a deal with CleanSpark. The deal raised $2.3 billion at an interest rate of 8.25% and saw more than $10 billion in demand from investors. Like many recent deals, the debt was structured with CleanSpark as the borrower and Meta providing a guarantee as the data center tenant. These deals have allowed hyperscalers to effectively rent their credit rating to smaller data center developers. In this case, we're seeing that investors are more than willing to keep lending to hyperscaler data center products, but they need a little more premium. The Financial Times has also reported that $18 billion of debt tied to an Oracle data center in New Mexico has hit stressed levels, quoted as low as $0.89 on the dollar. Oracle was already at risk of being downgraded from investment grade, and if that happens, it could trigger a wave of institutional selling. At this stage, it's just market jitters, but there are clear signs that people are on edge. Over the weekend, infrastructure-focused venture investor Meltem Demirors posted, hearing banks stopping all compute lending. Credit crunch is beginning. The post gathered 800,000 views and a ton of engagement, with many financial folks asking her exactly what she meant. Meltem later added, nuance: private equity firms and specialist lenders active. Investment-grade borrowers will continue to get funding, but more scrutiny. The long tail is drying up very quickly. Extreme divide between haves and have-nots. Compute is an underwriting game, and most credit quality is bad and getting worse. Still, investor Jigar Shah suggested it's a little too early to panic, commenting, Folks are freaking out about debt spreads widening for AI data center buildouts. No need to freak out. Yes, 2-3 year buildouts are hard, but many are seeing sub-25 MW data centers as a way to build 50% cheaper with 120-day timelines. Still, for those who are trying to keep an eye on where there are signs that the AI trade might be rolling over, This, rather than stock prices, is the corner of the market to keep an eye on. For now, however, that's going to do it for the headlines. Next up, the main episode. A new study from KPMG and the University of Texas at Austin found that when people work with AI, similar skills don't guarantee similar outcomes. Researchers studied more than 500 early-career professionals and found that the best performers consistently amplified the value of AI by guiding, evaluating, and refining its outputs. These top performers, called AI Amplifiers, weren't defined by what they knew alone, but by how they worked with AI. Learn more about what separates AI Amplifiers from everyone else at kpmg.com/us/aiamplifiers. Every AI coding tool on the market does the same thing first. It starts writing code. Blitzy does the opposite. Before writing a single line, Blitzy spends days reverse engineering your entire codebase. Thousands of agents ingest millions of lines, mapping every dependency, every undocumented constraint, every architectural decision made over the last decade. The result is a dynamic knowledge graph that understands your software the way a principal engineer would after 30 years in the building. Other tools guess at context with grep searches and markdown files. Blitzy never guesses. It builds true understanding first, then delivers over 80% of entire software epics autonomously. Validated, end-to-end tested, production-grade pull requests. That's why Fortune 500 engineering teams trust Blitzy with the codebases that matter most. See for yourself at blitzy.com. That's blitzy.com. Here's a harsh truth. Your company is probably spending thousands or millions of dollars on AI tools that are being massively underutilized. Half of companies have AI tools, but only 12% use them for business value. Most employees are still using AI to summarize meeting notes. If you're the one responsible for AI adoption at your company, you need Section. Section is a platform that helps you manage AI transformation across your entire organization. It coaches employees on real use cases, tracks who's using AI for business impact, and shows you exactly where AI is and isn't creating value. The result? You go from rolling out tools to driving measurable AI value. Your employees move from meeting summaries to solving actual business problems, and you can prove the ROI. Stop guessing if your AI investment is working. Check out Section at sectionai.com. That's S-E-C-T-I-O-N-A-I.com. This episode of the AI Daily Brief is brought to you by HyperAgent, where you run fleets of agents your team can manage together. Forget local agents and chat workflows waiting on your laptop to be prompted. HyperAgent deploys always-on agents in the cloud, doing real work across the tools your team already uses. Marketing agents turn competitor moves into landing pages. Sales agents enrich leads, draft emails, and updates the CRM. Ops agent chases the paperwork and tracks the budget. Every agent has access to shared context and follows your rules about scope and approvals. It's time you had agents that feel like teammates. Hire yours at HyperAgent. Get $100 in credits at hyperagent.com/AIDailyBrief. Welcome back to the AI Daily Brief. Right now, we are in the midst of a fairly significant transition in the AI debate. Specifically, of course, I'm referring to the regulatory debate, as this issue becomes, in the lead-up to AI, one with significant political resonance. Importantly, however, this week brings with it something unique in the context of that conversation, which is a chance to more directly understand and hear from China about where they stand relative to all these questions that are coming to dominate discourse here in the USA. Given how much of our regulatory debate is backstopped by assumptions about what China will or will not do, that gives this week the potential to be fairly significant. But if you thought the administration might take a weekend off in advance of those talks with President Xi, you were sorely mistaken. Over the weekend, President Trump announced a new AI-focused arm of the military, a search for a new AI czar, and a rebranding to address AI's flagging popularity. In a Truth Social post on Saturday, Trump railed once again against the, quote, many hoaxes that he argues are culminating in the, quote, decimation of AI. He declared, I, as President of the United States, will not stand by and let this happen. After talking about how, quote, wealthy and prestigious data centers were for the communities in which they were built, Trump announced, "I am forming the AI Force much like I did Space Force, which has been a tremendous success in my first term. To that end, I will be announcing in the near future the AI Tsar. Only high IQ individuals need apply. AI is the next industrial revolution or internet, but will be even larger and more impactful, possibly as much as 25% of our country's GDP. We are leading China and the rest of the world, and I intend to keep it that way." Following the announcement, the president spent the rest of the day workshopping a brand overhaul, writing in a post on X, many people think that the words artificial intelligence are inaccurate and very ineloquent relative to AI or artificial intelligence. A far more elegant and accurate description of this new phenomenon would be superior intelligence, SI, or extreme intelligence, EI, or supreme intelligence. The post even came with a poll, and after more than 230,000 votes, Superior Intelligence was edging out the others with 41% of the vote, with Trump later cutting Supreme Intelligence from the running, declaring it was likely losing because of its association with the Supreme Court. In that second poll, which at the time of recording has 178,000 votes, Superior Intelligence is just barely ahead of Extreme Intelligence, 51 to 49. Now, I probably don't have to go all that deep on the jaw-open, aghast reactions that this was what the president was spending his time on. With, by the way, those critiques coming from both the left and the right. Right-wing commentator Matt Walsh wrote, why the hell is the president focused on this? But at least a few folks were taking it a little bit more seriously. Signal on X wrote, once again, Trump's branding instinct remains undefeated and should never be ignored. He is absolutely right that the entire field is poorly branded. Taco Bell has spent decades teaching America what supreme means. It means more, better, etc. And artificial has almost always been used in a negative context in the American lexicon. Of course, even if one does take that branding mission seriously, no amount of rebranding is going to deal with the actual issues surrounding AI. So what did people think about AI Force? Well, as you might expect, many lampooned and derided the idea. And yet some weren't so sure it was worth dismissing out of hand. The argument for taking it seriously basically looks back at the example of Space Force, and notes that while it too had a lot of fun poked at it when it was first announced, when you strip away all of the nonsense, basically what it did was to consolidate satellite operations and defense between the Army, Air Force, and Navy, which were each operating their own assets and maintaining specialist personnel. It is not at all inconceivable that some sort of talent and resource combining, particularly around cybersecurity and cyber defense, might actually be valuable. In other words, hold aside all the other debates about X-risk and AI kill switches and things like that. One thing that's for sure is that we're going to be dealing with a new era of cybersecurity concerns thanks to advanced AI. Having a central body that provides support around cybersecurity to other arms of the military might not be a crazy notion. Now, speaking of kill switches, in other areas of the AI debate, kill switch bills are becoming a very popular policy. California Governor Gavin Newsom, for example, has signed an executive order to accelerate policy around an AI kill switch. The order convenes a working group to provide policy guidance within two months and includes other measures like a requirement for third-party safety inspectors. In a video posted to social media, Newsom said, "We're not waiting to act. We're going to speed up our work on substantial and responsible AI oversight before it's too late. We're going to do this thoughtfully but with urgent velocity. The stakes are too high to wait or delay action." Now hold aside the substantive merits of that policy or any thoughts you might have around Gavin Newsom himself. Newsom is, if nothing else, going to be a useful bellwether for understanding AI politics for the Democratic Party. Newsom is expected to be one of the leading candidates for the Democratic nomination for the presidency in 2028, and yet also comes from California, where he has in the past vetoed AI safety legislation, meaning that shifts in tone and the way that he justifies things are going to provide some interesting signal about where the winds are blowing. There has certainly been quite a bit of a groundswell around an AI kill switch over the past week. Andrew Yang presented this as an AI policy preference during an interview on CNBC. California Democrat Ted Lieu is renewing his push for a kill switch bill in Washington, and it's not hard to understand why something like this would be popular, although how much it does to address here-and-now types of concerns is a totally different question. We also got some new data center policy out of Virginia, where Democrat Governor Abigail Spanberger has signed an executive order placing new restrictions on data center construction in a state that has been one of the prime areas for data center building for many, many years now. As far as data center policy goes, it's fairly middle of the road. It's not a moratorium, although some of the policies could function as de facto bans depending on how they shake out. The order bans government officials from signing nondisclosure agreements, which as we've seen, is basically going to be a political non-starter from here on out. It also instructs the state government to expedite noise regulations and commissions a review of backup electricity provision across the industry. Alongside the executive order, Spanberger unveiled a new data center accountability framework for the state. Among other things, it removes the by-right approval that allowed data center construction on certain land without additional approvals. That streamlined approval process was a huge contributor to the creation of Data Center Alley in Loudoun County. Now, so far in terms of the response, it kind of looks like no one's happy with data center builders having new constraints and local opposition groups arguing that it's not going far enough. But then again, when it comes to democracies, sometimes nobody being all that satisfied is an indication that the policy has found the right sort of middle ground. Still lurking behind all of this conversation is the specter of China. The concern with all of these recent pause and slowdown types of talks is that if China doesn't agree to those sort of policies, they're effectively DOA and not actually doing anything other than cutting the U.S. out of the AI race. With that in mind, California Congressman Ro Khanna has urged China to join the AI slowdown. Khanna has sent letters to Chinese AI companies including Alibaba, DeepSeek, and Moonshot to join in a binding international agreement to pace the frontier. He's also convened an emergency congressional hearing in his capacity as the ranking member on the House Select Committee on China. That hearing will be held on Wednesday, just ahead of President Xi's state visit, which is of course what I was referring to when I said that this week could involve some actual progress, not just more undirected chatter. By way of background, Chinese President Xi Jinping will be visiting the United States between September 23rd and 25th. This follows President Trump's visit to Beijing back in May. The Washington meeting between the two presidents is reportedly set for Thursday, September 24th. And while AI is getting a lot of the headlines, the immediate economic priority is preserving the current trade truce, which is set to expire in November. Officials are discussing an extension and possible tariff reductions on U.S. agriculture and energy exports. And rare earths also remain a major pressure point, shipments to the U.S. falling 20% month over month in August. Still, AI is big on the agenda and was a key part of preparatory talks that happened on Sunday of this week, featuring key negotiators including U.S. Treasury Secretary Scott Bessent and his Chinese counterpart who oversees economic relations. Bessent heralded very successful talks with China Vice Premier He Lifeng and specifically noted the proposal for an AI incident alert system, which is sort of an AI version of the Kremlin-to-White House red phone of the Cold War era. Said Bessent, the U.S. has proposed that we have a notification mechanism between the two countries, and we want a shared vision of common goals and common threats. Now, on the one hand, this sort of bilateral communication could look like just the lowest hanging fruit, but Bessant basically said don't underestimate its significance. In a briefing with reporters, he said, we think that just like any cross-border activity, moving from opaque to more transparent between the number one and number two AI powers in the world is very important. So where does China stand on all these issues? Keep in mind, me trying to give you here a quick summary of how the country overall feels is no less incomplete and reductive than if my Chinese AI podcaster counterpart part was trying to give their audience a single point of view that represented the United States. But still, it does seem to be the case that there are some fairly key differences between the worries here and the worries there. One of the biggest differences is that the Chinese AI community has almost no, or at least almost no professed fear of existential risk. To the extent that there are any x-risk concerns, it's the Chinese government's fear that AI could be an existential risk to societal control and ultimately party rule. One of the few official policy positions came from China's Minister of State Security, Chen Yixin, who wrote on the topic a few weeks ago. Paraphrasing his communication, the New York Times wrote that, quote, Mr. Chen called for more party control over AI and stricter government oversight. This, he said, was necessary to protect political stability at home, to defend against increasingly sophisticated cyberattacks and misinformation campaigns from hostile forces, and to compete militarily with countries like the United States. Henry Gao, a law professor at Singapore Management University, said Beijing is going into these talks viewing AI safety not as a shared humanitarian mission but through an adversarial lens. He added that the government is making it clear that data sovereignty and political security will never be traded away for international safety accords. Last Monday, a spokesperson for China's Ministry of Foreign Affairs made their position on an AI slowdown abundantly clear. He said, the development of artificial intelligence concerns the common well-being of all humanity. All parties should jointly promote an open, inclusive, universally beneficial, and ethically sound approach to AI. Fearmongering and engaging in confrontation and malicious competition will only disrupt the global AI governance process and serve no one's interests. To that end, one of the biggest issues for China is Anthropic's stance on data security, stemming from a couple of incidents that barely made the news in the US. In July, researchers uncovered a backdoor in Claude code that traced Chinese IPs and reported on activity. Anthropic presented this as a necessary measure to block distillation attacks, but Chinese officials viewed it as similar to shipping malware to their largest tech firms. Following the revelations, Alibaba banned the use of Claude code, and government officials issued an industry-wide warning. More recently, in a risk report, Anthropic disclosed that Chinese government officials had been using Claude with no regard for privacy. They reported the officials uploaded sensitive government documents, including details of Chinese military operations. On Saturday, a social media account linked to Chinese state TV warned that Anthropic was sharing data with U.S. intelligence agencies. Part of the claim was that Anthropic had altered their terms of use multiple times without proper warning. Basically, heading into this week's meetings, it seems that setting the terms of engagement for AI-powered espionage will likely be a far higher priority for Chinese officials than anything to do with existential risk. The other big difference between the two AI superpowers is how the technology is impacting the broader economy. In the U.S., to put it bluntly, the AI buildout has been keeping the economy out of recession. By most accounts, AI CapEx has been the core driver of economic growth over the past year. In China, however, the economy isn't seeing any significant pickup from AI. The New York Times writes, economists in China, even those closest to the state, have openly warned that the government is pouring too many resources into a technology that creates relatively few jobs while doing too little to save the broader economy. Now, the reason that the Chinese economy is flailing has nothing to do with AI. The issue is that there has been no sign of an AI-driven recovery to help the struggles of the past few years. Youth unemployment has reached 18.9%. Domestic consumption is in a full-on downturn, with automobile sales falling 20% in a year, and housing continues its decline with another 14% annual drop. Li Daokui, a former Chinese central bank and economics professor, said that the economy is running too cold. And a booming tech sector can't lift a larger base. Economists have proposed quintupling basic pension payouts to shore up a cratering consumer economy, meaning that right now Chinese officials are dealing with an economy that is closer to crisis than carving up the spoils of an AI boom. Now, for some, none of this should really matter when it comes to U.S. policy. In an interview with CNN, for example, Microsoft AI CEO Mustafa Suleyman said, I don't think we should use China as the boogeyman for not making progress on our own efforts. He basically argued that each AI lab is responsible for their own safety measures within a pluralist AI system, and said that he didn't view regulations as negative but instead should be seen as a way of creating shared norms and standards to increase safety. It shouldn't have to slow us down. Whether that view becomes more widespread remains to be seen, but what's for sure is that this week will advance this conversation in some way, shape, or form, and we'll check back in at the end of the week after those talks have actually happened. For now, that's going to do it for today's AI Daily Brief. Appreciate you listening or watching as always, and until next time, peace.