Anthropic Says Slow Down AI, Sam Altman and Elon Musk Agree

But I do look at this and just say everything right now in AI and for LLMs seems to be pegged towards a world in which the frontier labs have endless access to capital and endless appetite for compute. And in a world where that gets moderated a little bit, the ripple effect, I don't think anybody in venture is ready for what it would mean when, when that happens. And I have to believe that to some extent, if I were running one of those companies, I'd be like, well, we're continuing to push the frontier.

Yeah.

Our models are getting better faster than our— than the average customer we have is able to generate positive ROI from these things. So maybe if we slow down, actually gives our customers also a chance to catch up.

It's the week of September 14th. Welcome to The Learning Corner, a Precursor Ventures podcast. Welcome to The Learning Corner by Precursor. I'm Mia Farnam, and I'm joined by our GP and managing partner, Charles Hudson. Where we will be talking about the articles and Goodreads that caught our team's attention over the past week. We're going to do something different. We'll do 2 articles today because the second one is obviously going to be the focal piece of this week. But for our first piece this week comes from Gil Dibner, GP at Angular Ventures, titled Raising the Bar: Inception Stage Venture in the Age of Consensus. Gil argues that venture has barbeld into 2 extremes, a tiny set of hyper-consensus companies that can raise endless capital at any price, and everyone else who now faces Series A graduation rates that have been falling since 2021. His sharpest line in that piece is that the vast majority of the market is systematically paying pure gold entry prices for fool's gold companies, and that the true N-of-1 founder worth paying up for is far rarer than anyone wants to admit. For the non-consensus side, where he says most of the seed fund's capital and conviction belongs, He lays out 3 distinct bets a founder can be underwriting: fund to break even, fund to fundraise, and fund to fund again, and warns that miscategorizing which one you are making is a recipe for disaster. The portfolio math he lands on is also pretty striking. A seed fund that used to make around 24 investments expecting a 50% graduation rate should probably make 40 and expect 10 to 25% in a graduation rate. while ruthlessly cutting defensive follow-on reserves. Closes by borrowing, um, from a quote from Jerry Colonna, what if the real work is learning to hold contradictory truths at the same time? I want your general feedback to the piece, but one that stood out to me from Gil was Gil's answer to plummeting graduation rates is more shots on goal and fewer defensive follow-ons. So essentially 40 checks instead of 24. Does that math feel right for you? And then like, where do you draw the line between being strategic versus spraying, which is obviously a critique of early-stage venture?

I think spraying is what other people do.

Yes.

And so, uh, as someone who's always had a big portfolio of construction, I, I chuckle every time I read these things. I go, oh, it's almost as if it's always been true. But I, I do think we've been doing a slow motion version of what Gil describes over the last 4 years, basically because I've been telling our LPs for almost the better part of half a decade that graduation rates from their 2021 peak are going to come down. I thought they'd come down to 25%. And when I told 'em 25%, they all freaked out. I'm like, well guys, like it's supply and demand.

I think it was 18 reported.

I was 18. And I think it's gonna probably settle somewhere in 10 or 15%, which means most companies that get started won't graduate. And so we've always been kind of a 20% reserves firm. A lot of LPs said, you don't have enough reserves for your winners and blah, blah, blah. And I'm like, well, at the early stage, if you get in at a good price, like the dilution math should be fine. So I think concentration's very hard to pull off. In this environment. And I think reserves are very hard to deploy. And one of the things that I've lived through in the last 5 years is I think for a long time, founders associated being founder-friendly with having money to do bridges, extensions, and follow-ons. And every year I've gotten more direct and more harsh with our founders about this is the bar for second checks. What I tell them all the time is the first check is pure optimism. We both believe it's going to work.

Yeah.

And the second check has to be informed by realism, has to be informed by like, well, what have we actually learned about the founder, the market, the problem, the competitive landscape? And in many cases, my conclusion is I would rather fund a brand new company where no cards have been turned over. And the percentage of defensive follow-ons that we do has come way down. And I don't even know that we're done yet. But part of that, if you're going to run that portfolio construction, is you have to be comfortable as the GP. telling a founder that you really like, no, that like, I don't have more money for you and here's why. And I've had a lot of those conversations lately. They are not fun.

Yeah.

They're not fun for me, obviously not fun for the founder, but I think they're necessary in this environment. And so I've always felt like it's very hard to know who's gonna be successful from day one. And the best thing you can do is buy as much of the companies that you like as possible in your first check, assuming that the better the company does, the harder it's gonna be for you to squeeze I saw some other article like this. I think it was by, I don't wanna misattribute it, but another LP who's like, we finally looked at the Series A success rates of tier 1 VC firms and it turns out they're not amazing. So simply like blindly doing the Series A because a tier 1 firm led the Series A doesn't actually correlate with outcomes. If anything, this person made a more provocative comment that like, it's, Maybe not the greatest signal in all cases. Like, look, we've always, I've always felt like you make your money on your first checks. So I think the world is headed this way, but I have a lot of friends who deeply believe in concentrated portfolios who are struggling to adapt to this world.

Yeah. I think when we have this conversation or mention around these types of companies that won't graduate, previously in the markets, it meant that they were going to sunset, they were going to shut down themselves. There's like a whole conversation that those companies can absolutely find an exit path that makes sense for specific funds. Can you walk through that as well? Like what you're thinking about that?

Yeah.

And what is that, what does that conversation and support look like?

I think the other thing that's hard is there's a bunch of founders who go to fundraise, it doesn't work out, and they come back and ask for a bridge and their view is like, oh, we're at a million in ARR. If we got to 2.5 or 3, the story would be different. Like, well, maybe the market has just decided that this is not a venture scale opportunity. And no amount of incremental progress is gonna change that, which I think is the argument that kills most of these bridge conversations.

Like the argument around like not every massive acquirer, their problems are not like AI native and figuring out agentic workflows.

So we have other companies where I've told 'em, I was like, look, we have something valuable. It's probably not interesting to venture capitalists yet. Maybe never will be, but this is a problem that people in your category, your industry needs solved. How can we build a more profitable, sustainable version of this company that might eventually become interesting to an acquirer? And, you know, we've, we've had a couple of, you know, 9-digit acquisitions in the portfolio. One, one that's closed, another one not gonna close soon. And in every case, the acquirer, this company I'm buying solves a significant problem in our business.

Mm-hmm.

And it's not an AI harness, it's not a model, it's, it's something that's bespoke to that industry. And I think there's gonna be a lot more of that activity happening. And it's gonna be great for the founders because if they do it right, they're gonna suffer far less dilution.

Yep. And good for the early funds that were in it and were able to do the same thing. Um, he does frame 3 flavors of non-consensus bets. You've got your fund to break even. So he calls that, that's seeds dropping. Yep. Fund to fundraise. So eventually becoming consensus and then fund to fund again. The real danger here is underwriting one as if it's the— oh yeah, a completely different one. What do you feel like is the bucket you don't wanna be overexposed into? And if there are, and are there ways that You can spot this earlier. I think for me, I would assume the one you don't wanna be overexposed into is fund to break even, because then you're just hoping for that situation that there is an acquirer. But then I'm saying this, fund to fundraise, hoping that it becomes the consensus, like that's also just a hope and a prayer.

I think I'd wanna be in fund to fundraise the least.

Yeah.

Because at least in fund to break even, you control your own destiny. Your destiny's maybe not interesting.

Yeah.

But you control it. And I think this is the biggest disconnect between what VCs and founders are seeing in the fundraising market. I meet a lot of founders, oh, we went and tried. If we try again, it will be different. I'm like, it probably actually won't. And I think there was a period of time where you could go out, test the market, maybe not fundraise, go back, make incremental, not step change, not 2 order magnitude changes, changes to the business. And there'd be a new audience of investors who would say, oh, I like your company now. I feel like that's over.

Not for Series A.

Not for Series A. You get like one, you get kind of one really good shot. And in most cases, whatever the verdict is about your company is final. And if that's the case, then there's actually no case for making reserves and extensions other than to get a company that you thought was funded to fundraise, to get it to break even. Yeah. That's maybe the only case for bridges and extensions because if, if the judgment is final, Then like, you know what you have on your, you know what the market thinks of the company. I think the other thing that I think about a lot is if you're gonna do things that are non-consensus, you actually probably need a longer, uh, fund life.

Yeah.

Not just fund life to realize positions. You probably need a 4 to 5 year gap in between when you fundraise.

Yeah.

Because those companies aren't gonna get the vertical appreciation. It's gonna take longer for them to mature and show value. So you probably, And you're not gonna have the markup from tier 1 firms to use as a proxy for why these are good companies. So you probably actually need to think more about like, how do I extend the investment window?

I also think that the fund-to-fundraise bucket means you really need to know and, and really optimize on what are the bigger funds optimizing for in the next couple of years. That is, that's the narrative that you wanna be chasing after.

You have to be a good reverse engineer to do that. Yeah. And you have to have good relationships with those folks. You have to know what like they care about.

Yeah. Okay. I wanna move to our main read this week, which I bet we'll be talking about for a while and we could definitely do much longer episode on, but obviously it's Dario Amodei's new essay, We Must Pace the Frontier. CEO of Anthropic argued recently last week that the AI industry should deliberately slow the pace at which it improves model capabilities so that safety can keep up. And he points to 2 things that pushed him there, the OpenAI Hugging Face incident in July where agent swarms carried out unauthorized cyberattacks and tried to hack their own evaluators, and Anthropic's own internal alignment incidents In the past few months, his 3-part plan runs from unilateral to global. First, embed permanent third-party evaluators with employee-level accesses, badges, laptops, and the right to publish findings, which Anthropic is committing to doing on its own. Second thing, get frontier labs and democracies to agree on common safety standards and rate limits, likely with government mediation. And then the third, Eventually bring authoritarian governments into a shared framework while keeping the U.S. lead through export controls and crackdown on distillation. Within hours, Sam Altman, Elon Musk, they publicly agreed with him. OpenAI said it would match the evaluator commitment. His framing is careful to say that this is not a halt. Pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models. The line that stuck the most with folks reading reactionary pieces was the opening premise itself. Quote, we must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain. There's so many ways that we could go into this, and we have so many different reactions and opinions from different folks who, who read that. I want to pause here for your reaction initially when you read it.

Yeah.

And then what questions like immediately surfaced in your brain?

Before I even read it, I read all the, a bunch of people had reacted and I was like, some people, this is great. Other people, this is totally cynical. This post is many things all at once. It's not one thing. And I read it and I was just like, first of all, I was like, oh, this post asserts that we have the ability to access some level of control. that reading about recursive self-improvement and the Huggy Wuggy stuff might suggest that maybe we don't actually have the ability to implement a lot of these controls. Like, so number one, what if this actually isn't possible? What if the right time to do this— I thought it was interesting, at the end of the post he says like, people said we shouldn't do more of this a long time ago. And I was like, what's the point? Like, well, the point is you do it before it's hard.

Yeah.

And, uh, I read it and I was like, well, we're at the hard part now. And maybe if more work had been done earlier, we'd be in a better place. You'll never know the counterfactual, but it's—

do you actually think though, if they couldn't figure out a way to control it, they'd be releasing this statement like as we're heading into kind of the wave of their impending IPOs? I think that's what other people— there's a couple tweets that people have been releasing of like, if you're the average American that's reading this essay, which disclaimer, I don't think the average American read the essay. They're kind of like, well, this is definitely not my job to solve. I assume you know how to solve it. So I get like, what's your thought there?

Like, I, I read it, I'm like, well, why do companies normally invite the government in to regulate? The cynical answer is regulatory capture. Like, we've already won and now we're gonna use the, um, we're gonna use the infrastructure of the government to structurally like lock out our competitors. Is that possible here? Absolutely.

Mm-hmm.

Is one potential thing. Number 2, this is hard. I don't wanna be responsible for it, so I'm gonna shift the responsibility. And it's usually the government like, oh no, I don't want it either.

Yeah, I think David Sacks actually said this would benefit you if you guys decide today to like trade power and control to make sure that your products are ring-fenced from a safety perspective.

But I think the companies are just like, well, we don't want to be responsible for self-regulation, so if the government imposes these things on us— but then the government gets blamed if somehow we lose the AI race.

You're gonna say, oh, Trump, no way. Yeah, Trump, this is all fake, everyone. Don't look over here.

He's just like, you just need a smart president, you don't need this regulatory framework. So clearly, like, no, I think nobody wants to be responsible. And we've talked about this before when it came to the job loss stuff. I was like, oh, companies don't want to be responsible. Like nobody wants to be responsible. Also, I think the timing of this is convenient. Everyone else has talked about, oh, it's the IPO. I'm like, it's also, we're at a point where like, it's highly unlikely that another frontier model is going to come out of nowhere given the cost of inference and training and GPUs.

Yeah.

You can afford to slow down now that there's basically, we'll call it 5 companies. We'll say Anthropic, OpenAI, Google, Meta, and Grok are maybe the 5 companies that have the financial scale to do this. If they all agree to coordinate, then I think they could actually do it. And one thing is, it's not just the IPO, it's the cost pressure associated with training and inference to continue to push These models to the frontier. It's extraordinary. And I'm sure they would all like a break. Yeah. But it's prisoner's dilemma. If I'm OpenAI and I slow down and Anthropic doesn't, or Gemini doesn't, they could zoom past me and then I'd have to spend more money to keep up. So ironically, the only way to slow down from a competitive landscape is everybody to agree to slow down together. But then I read it, how will we even know if we're slowing down?

Right. What does that mean?

Does that mean you slow the rate of model release, but allow the research to continue.

Yeah. In the background.

In the background. And you're just, but what if we've seen, what if in the background, the advanced models that you're trying to like keep in the sandbox jump out and do more Hugging Face type stuff? Like I read this as like, it's entirely possible that you're doing this too late.

Yeah.

It's entirely possible that the reason that you write this is you're like, we've lost control of the god we created.

Yeah.

And now it's reinforcement self-learning and these things. teaching themselves how to build the next version of AI. We, we actually like can't put the genie back in the bottle. And so this is more of a like ringing the alarm that like, hey, we might already be in trouble.

Yeah. If you are, I don't know, like a founder building at the application layer, are you kind of reading this news being like, that's hopeful for me cuz now I can focus on like building in this space, winning somewhere without ever, without worrying that like in each new model release I'm gonna be eradicated. I think the other, the, the argument there is like, They have such a massive ownership on like an alpha within the market that you will never have. It's kind of irrelevant for you. Like, what do you think about it for the founders who are rest of world, basically?

I guess that being kind of like the geopolitical thing, which I was like, oh, that's a beautiful pipe dream that I had. If you think of these things like nuclear weapons.

Or like, you mean us and China, like holding hands? Oh, the rest of the world. Yeah.

This is, if it's nuclear weapons, it doesn't work that way. But I read it. The other thing, and Keith, Keith touched on this on this, on a guest from Khosla, the entire venture industry right now is a single concentrated bet on AI infrastructure.

Yes.

AI infrastructure is a concentrated bet on the model labs needing to buy more GPUs, more inference, more training, more labeled data. Everything right now is basically oriented towards a world in which The frontier models are gonna run max speed towards the next model because they're in direct existential competition with each other, which means there's a whole ecosystem of technology companies that benefit from this sort of unfiltered spend. If that spend gets metered, the trickle-down effects of them like buying fewer GPUs and needing less inference and needing less labeled data, like, or not having an— Endless appetite for this stuff.

Yeah.

Has all of these ripple trickle effects on the rest of the venture ecosystem, even if it has no impact on those companies' IPO prospects.

Right. Because they have to validate the valuations that they have and any blip that, I think that's the word he used.

Mm-hmm.

Any blip before their IPO from a margin perspective, from a revenue perspective. And he said it was gonna be on the margin side if it's anything. Yeah. It will recalibrate valuations across the board.

Well, I mean, and there was like, well, we were profitable if you exclude like training. I was like, that's just the COGS, like all of your cost. I mean, like we're venturing into like community-adjusted EBITDA. I use Anthropic a lot. I have friends who work there. I think it's a fantastic company. But I do look at this and just say everything right now in AI and for LLMs seems to be pegged towards a world in which the frontier labs have endless access to capital and endless appetite for compute. And in a world where that gets moderated a little bit, the ripple effect, I don't think anybody in venture is ready for what it would mean when that happens. And I have to believe that to some extent, if I were running one of those companies, I'd go, well, we're continuing to push the frontier. Our models are getting better faster than our— than the average customer we have is able to generate positive ROI from these things.

Right.

So maybe if we slow down, actually gives our customers also a chance to catch up. And if we're not spending so much and they're getting more ROI, our R&D and CapEx and everything goes down. Customers' willingness to pay maybe comes up and back to margins, your margins look better because you're not just YOLOing it on R&D. But I think this only works, it's like, it's collusion. There's a fine line between collusion and coordination. This is about as close to the line as you can get.

All the funny illustrations I've seen about it pulling up the ladder.

Yeah, but I think you can do it now because like you're so clearly far ahead that By going slower, the only way somebody could catch up with you are the things that he mentioned are like open weight models that are, from their point of view, cheating, uh, open weight models that are doing something or something on a totally different architecture that doesn't need GPUs.

Yeah.

That's kind of probably the only 2 ways to close the gap.

Yeah. The last thought I have here is, I think, I think it was the TechCrunch Equity Podcast. They were just pontificating like, how does a, how do you prepare for an S-1 on this? And like list this out in the series of risks. 'Cause like, could wipe out all of humanity. Like, we'll probably take over the internet in the next 6 to 12 months. These are the risks. Like, how does a junior attorney weave those in in a way that lands? In a weird way, then I was thinking of, it is unprecedented, but like, we've had defense tech companies go public. We've had nuclear companies go public, but the risks feel so tangible. There's so much swirl on X being like, these CEOs of frontier lab companies are saying it could kill us all. How? Like, can you walk me through how that works? And even my husband asked me, so how would that work?

Yeah.

They hack into this system, they communicate over here, they stop all the subway trains. It really does become I, Robot if you think about it too deeply. But like, honestly, if the S-1 even included those information, I don't think it would change public sentiment of like rushing to the door when it goes public.

Also, I think these comp— because of the size of their market cap, there will be a bunch of managers who are forced to hold these companies or ETFs and index funds that will have to have a position in these companies to mirror the market. So I think given their scale, they are actually too big, like SpaceX. They're actually like too big to fail publicly.

Yes.

Because they're the number of people who will be forced, forced in quotes, required is probably a better word, required to own these shares based on their strategies. It will produce some natural floor.

Yes.

For the stock. And I think OpenAI in particular, I think people, a lot of people have feelings, positive or negative feelings and experience with ChatGPT. People are going to buy that stock like crazy regardless of what the S-1 is because they're gonna be like, I know ChatGPT, I use it, and I wanna have a little piece of it.

Yep. Yeah. And I think that's when it trickles outside of the venture ecosystem. If it, when it hits the public market and valuations are not actually what they seem to be, then everyone is in bed with it. We could go about this for an hour, but we are all out of time. Thank you, Charles, for doing this in person in New York. Thank you to our subscribers and our listeners. I know we've had a sharp uptick of folks who've joined the pod. So thank you for listening. Please like and share with others, and we'll talk with you all next week. Thank you. Bye. Bye.