The Air Street State of AI Report 2025

Live from Schenectady, New York, it's the Metrics Brothers, Growth and CAC.

And I'm Growth, better known as Ray Reich, founder and CEO of Benchmarket.

And I'm CAC, better known as Dave Kellogg, EIR at Balderton Capital, independent consultant, and the author of Kellogg.

And together we are the Metrix Brothers.

And we go together like David and Goliath.

Wait, Cap, since you're Dave, that is David, that means I'm Goliath. Holy cow, I'm so excited.

Hey, easy there, Growth. You're not Goliath. The topic of today's episode is gargantuan. That's Goliath. We've got a Goliath of an AI report on our hands here. The Airstreet Capital State of AI Report.

It is a monster, Dave. 303 pages of AI delight. What a better way to spend my European vacation.

You shorted them 10, Ray. It's 313 pages of AI delight. And I'm so glad that you could enjoy reading it in Portugal in preparation for this episode. And today, somehow, we're going to try and summarize this beast and get some key takeaways.

But before we do that, Dave, let's hear a word from our sponsor. The Metrics Brothers is proudly sponsored by Benchmarkit, the B2B AI and software industry's largest, broadest, and most contextualized benchmarking index. Dave Kellogg and Ray Groth Wright have put the Metrics Brothers podcast on hiatus for the remainder of 2026. All episodes we have ever recorded are still available on your favorite podcasting app. You can reach Dave @kellogg and Ray on LinkedIn @rayreich. Thank you so much for listening to the Metrics Brothers podcast.

Okay, Ray, this thing is a doozy, as we talked about, 313 pages published on October 9th, 2025. So a little bit more than a month old or coming up on a month old. So fresh off the press. This is an annual report and it dates all the way back to 2018. So, the reason we're doing this episode is, I can't remember who, somebody told me about this thing, maybe on social media. I went and found it and I was like, holy cow, we need to get the word out on this thing. It's a report that they've been publishing for 7 years and somehow I'd never encountered it. The report is written by a guy named Nathan Benaish, a solo GP at Airstreet Capital. He's got a PhD from Cambridge in computational and experimental cancer research. So this guy is going to know a lot about not only biology, but AI. Airstreet is a London-based VC firm that invests in AI-first companies. He writes a Substack called Airstreet Press, and he's pretty prolific. There's a lot of content there. So he writes from a stunningly broad perspective. This 313-page report has deep sections on AI research, AI in industry and commercial application, politics, regulation, economic implications, geopolitics, safety, the identification and mitigation of potentially catastrophic risks. In the middle of this thing, there's 20 pages plus of a survey of 1,200 AI practitioners. And there's also predictions. So this is one of these reports, Ray, that I want to and have already read kind of n times. It just sits on my desktop. And when I get bored, I open it up and read it. Or when I'm on an airplane, I open it up and read it. Because each pass, I understand incrementally more. Thoughts, Ray, on this thing?

Yeah, I mean, the research, which is the first section, I got through about 15, 20 pages on large language model research, what models are performing, benchmarks. So my advice to the reader is, if you're not an AI researcher or developer, maybe you start with the industry or commercial application section, go and read the survey. If you're a big geopolitical person, go read the politics and the predictions, but kind of name your own journey because some of it's just so deep if you're not an actual AI researcher, AI scientist.

Ray, do you mean that sentences like, researchers show that Muon expands the compute time Pareto frontier beyond AdamW, the first optimizer to challenge the 7-year incumbent in large-scale training? Did you find that impenetrable?

Yeah. Is it happy hour yet?

Yeah. So I think Ray makes an important point on this survey, which is if you open it up and try and do kind of a death march through it, you will not survive the research section, probably unless you have a master's in AI or maybe a PhD. So I would definitely recommend looking at it section by section and don't go linear and get depressed in the research section and give up. There's plenty of great stuff after that that is much more accessible content. I would say that the amazing thing here is that having spent a lot of time in all the sections, that it's as pithy in research as it is in geopolitics. I mean, this guy, and I'm going to say this guy because he's a solo JP, the report has 3 co-authors who are on his team, but I do think he's kind of the lead on this. He's a polymath because he's deep and broad. I mean, he's hitting all angles. And that's what makes this report so beautiful is it's comprehensive kind of period. I will just say very comprehensive, but that's not necessary. Comprehensive means comprehensive. And boy, is it comprehensive.

Yeah. Well, let's, let's, why don't we go ahead and jump into it? Because I think it started with an executive summary, Dave.

Yeah. So, so actually, before we do that, Ray, hang on. So the question you might be wondering as well as the audience is, okay, what are we going to try and accomplish with a 313-page report in 20 minutes or 22 minutes, which is our target episode length? Um, I think there's 5 things we're going to do today. One, we're going to make you aware of the report. I think this thing is as seminal as Mary Meeker's Internet Report, which was first published in 1995. So us telling you about this today is like hearing about Mary Meeker in 1998. Kind of, there's a few years into it, but everybody hadn't heard of it yet. By 2000, 2001, everyone had heard of the Mary Meeker Report. So even though we're 7 years in on this thing, I think it's just because Nathan started earlier. And for whatever it's worth, content-wise, I actually think it's better than the Mika report. I think this is an excellent report. So one, we've done our first job, which may be the most important one, which is make you aware of this thing. Two, and we're gonna tell you a little bit about how to navigate it because you can get lost, as we've also done. Two, we're gonna review the executive summary and predictions, which Ray was champing at the bit to do.

3.

Ray's gonna review the survey for us, which is a part of this report. Then we're gonna do a couple of fun pull quotes, and then we're gonna review the 2025 predictions. Sound good, Ray?

And we're gonna do all that in 22 minutes. We better get going.

That's right, we've already burned 7. So executive summary, I'm just gonna skim the summary, but he summarizes in 4 areas. One, research, which we won't spend a lot of time on, but just so we all know what's coming. Reasoning defined the year, with OpenAI, Google, Anthropic, and DeepSeek trading leads. And then open models improved fast and China's open weight ecosystem surged. So good thing to keep an eye on. The proprietary closed models still have an edge, but the open source is coming on strong. Industry, real revenue arrived at scale. AI-first companies crossed tens of billions. NVIDIA ripped past $4 trillion in market cap. And this is a great benchmark, 90% ownership of citations in AI research papers. So what a cool metric. He also says, and this gives you an example of how pithy his writing is, circular mega deals funded huge buildouts. Wow. What was that, 6, 7 words to describe, you know, hundreds of billions of capital moving around. But I think it's correct and pithy. Power became the new bottleneck. You heard about it here on the Metrics Brothers. We did a whole episode on gigawatts and Manhattans to try and make those numbers real. And as we also mentioned, the AI revolution is compute-bound, which in turn becomes power-bound. And it's the new bottleneck as multi-gigawatt clusters moved from slideware to site plans. Again, more pithy writing. Politics, the AI race heats up with the US leaning into America First AI and the whole national AI policy. Regulation takes a back seat in the face of turbo investments. AI goes global became concrete with petrodollars and national programs funding gigantic data centers. So all stuff we all hear about. And finally, on safety, I think one of the more alarming quotes from the report was this one, the second bullet under safety: external safety organizations operate on annual budgets smaller than what leading labs collectively spend in a single day. So just think about the size and momentum of this industry versus That of those trying to keep us safe. And it's a little bit startling. So, Ray, any thoughts on the executive summary before we talk about the 2024 predictions?

No, I think we should jump into it because a lot of those executive summaries actually go into his 10 predictions. Things like a major security breach is going to get the UN involved. But I don't want to jump too far ahead. Okay, Ray.

So I'll keep going then. So on slide 10 of this report, he reviews Look, I do an annual predictions post on Kellogg. I think an important part of the predictions process is to do a review, and Nathan does one here. So on slide 10 of his report, he's reviewing his prior year predictions for the current year. And, uh, on the first one, he gets a yellow. He was thinking there'd be a $10 billion plus investment from a sovereign state that would invoke a national security review. Didn't quite happen. Um, the other thing I like about this report is They make big concepts super concrete. It's kind of like being really good at writing OKRs, right, where you want to take this high-level objective but translate it to a highly measurable thing. So he takes these big macro trends and turns them into super tangible predictions, like an app or website created solely by someone with no coding ability will go viral. Boy, that's a specific prediction. And did it happen? Yes. Formula bot built entirely using Bubble exploded to 100,000 visitors overnight. Next prediction: Frontier Labs implement meaningful changes to data collection practices after cases begin reaching trial. Yes, Anthropic did a $1.5 billion settlement with authors, deleting works and shifting to legally acquired books. So that one happened. Early EU AI Act implementation ends up softer than anticipated. after lawmakers worry they've overreached. He gave himself a no on this because the commission is really phasing obligations and leaning on voluntary code of practice as opposed to the harder binding rules. But in spirit, I think he was more right than wrong, which is the letter of the law is quite strict, but they're being kind of gentle as they approach enforcement. Next one. An open source alternative to OpenAI, O1, surpasses it across a range of reasoning benchmarks. That's a yes. Apparently DeepSeek R1 outperformed OpenAI's O1 on key reasoning benchmarks. Next, challengers fail to make any meaningful dent in NVIDIA's market position. Yes, NVIDIA remains dominant. Level of investment in humanoids will trail off. There's a no on this because the level of investment in humanoids actually doubled in 2025 to $3 billion versus $1.4 billion a year ago. Strong results from Apple's on-device research accelerates momentum around personal on-device AI. No. While Apple did a rollout of these things, I don't feel like it's taken off yet, or he doesn't feel like it. He gave himself a no. A research paper generated by an AI scientist is accepted at a major ML conference or workshop. Yes. An AI-generated scientific paper entitled The AI Scientist V2 is accepted. And then finally, a video game around interacting with GenAI-based elements will achieve breakout status, and it did not happen. So I know there's a lot of detail in those, but I like that he takes all this macro and distills it down to like these really tight OKRs, measures them, and then assesses them. Any observations on that one, Ray? I think you had one about humanoids.

Yeah, the 2 that jumped out at me was the humanoids. So this was just in the news yesterday with the trillion-dollar pay package. that the board of directors at Tesla approved for Elon Musk. And one of the 3 big components of that was to sell 1 million of the Tesla Optimus, the humanoids. So there's real money being put. In fact, Tesla just announced building a whole 2 new factories dedicated to Optimus. So a lot of investment going on there. And the other thing that I thought was really interesting for this report was the fact that like Anthropic did settle that $1.5 million with a billion settlement. But at the same time, content creators, including the major studios, movie studios, are now doing partnerships with OpenAI and Anthropic. So they're suing them, but now they're getting on the wagon because it's moving so fast.

Yeah, and those partnerships might actually in some ways be kind of settlements, if you will, right? Where we're saying, hey, if you partner with us, I mean, basically content producers, and I'm one, want to get paid. Right? When I use ChatGPT and I see it citing me in an answer, it's nice to get the citation, but I got zero web traffic. Luckily, I don't have ads on my site, but if I had ads, then, you know, all that stolen traffic or front-run traffic means less ad revenue for me. Thankfully, it's not my monetization strategy, but I certainly understand why content providers are super sensitive to these issues. Ray, I think you wanted to zoom us in and talk about the survey here because I know you love surveys.

I do love surveys. That's what I do all day long. And this is like the 6th section of the report. I think it starts around slide 283, Dave. You can double-check me on that, but 1,183 participants. 49% of the participants were from earlier growth stage. 21% were from public companies and 13% were from academic. What I thought, I've never seen this before. If you look at the profile of the participants, 46% had their master's and another 27% had their PhD. So this is not your normal general public participants.

Yeah, there's a lot of selection bias here because you have to have heard about the survey and opt yourself into it. But, um, but yeah, it's going to be a survey of highly engaged, highly educated AI professionals.

But only 22%— I'm rounding up now— were actually had AI in their title, like AI researchers, AI developers. So the things I want to highlight, and we've talked about some of these on, like, our ROI episode, 92% of the participants said that they personally have got increased personal productivity from using AI. Then on slide 287, they talked about most surprising use cases. And a lot of people said it was coding. Even people who weren't coders have used generative AI to do coding. So that was one of the most surprising use cases. And then slide 291 said that most people are seeing slight to moderate budget increases. That's true in most of the surveys I've seen. And Dave, this was surprising to me. They asked people what the biggest use cases were for them. Coding. Coding was the number one use case for 72%. So, you know, the cursors, the Claude coding, the lovelables. They are the number one use case that this population said, and number two was content creation. But on the I was surprised side, one of the lowest was customer service. Dave, only 14.7% of these 1,200 participants said that they're using AI in customer service. So those are some of the big themes and trends I saw. And then the last thing is on slide 300. It's what do these participants say? Are the emerging trends to look for in 2026 and beyond? 55%, the largest cohort, said it was agentic AI agents. And the other thing, and why I recommend people try to read that research section, is that 45% think that small language models, not the big LLMs like OpenAI, that there's going to be a huge shift to using smaller models because they have a little more efficiency and might have some cost efficiency also.

Yeah, let's talk about 2 things you raised there, Ray. One, let's talk about small language models. You know, I'm not an expert in AI, but whenever I look at this, I always feel like there's an enormous amount of brute force being thrown at a problem, right? Like massive training datasets, huge models. And one has to wonder, and we've talked before about the bitter lesson and how general-purpose models tend to beat highly tuned specific purpose models over time, which tends to reinforce the— there's a log. I'm a fly fisherman, as you know, Ray. And whenever you go fly fishing, you look at like an insect hatch or you look at a salmon hatch and you're like, or a bunch of salmon eggs. And a salmon egg is about a 1% chance of becoming an adult salmon. I don't know if you know that.

Wow.

And I think insect hatches are even worse. And there are many times where nature just throws numbers at problems. That's always what I look at whenever I see an insect hatch. I'm like, oh, we're just throwing numbers at a problem because they're getting eaten by fish everywhere. But enough of them get through to propagate the species. And when I think about AI and training, that's what I look at it like, we're just throwing numbers at this and that's what creates the energy bottleneck. And somehow if we could find an approach where we weren't throwing so much brute force at it, for example, small language models versus large ones to limit the scope to a certain domain, that one would hope that would be a more efficient way of doing it. And I was happy to see that in the survey. So, 'cause intuitively, without being an expert, that was my hope. And it makes sense in the early days of a wave to just throw muscle at it to prove you can do it and then tune it later, right? Make it more efficient later. That makes a lot of sense to me. The other thing I wanna talk about, which you did, which you talked about, was kind of coding, and it was kind of a surprise use case. And the issue is, one, I think it tells us a little bit about the audience, right? Something like 45% have a master's and 35% have a PhD, right? So these are highly educated people.

Yeah.

I erroneously earlier said highly educated AI people, and that was wrong. I'm glad you corrected me. They're just highly educated people. But the thing you talked about was coding. And on slide 107, he's got a great kind of one-slide evolution of coding, just reminding us that, you know, in June 2021, we had the GitHub Copilot launch, right? So kind of the pair programmer concept, hey, it's gonna help you. By 2023, we'd gone from autocomplete to conversation, right? So we're basically having a conversation with the tool, and the tool is then writing code, right? And then today, we have AI-native IDEs that are more like a full-time engineer than an assistant. They're kind of writing code with minimal oversight. It's almost like a manager-employee relationship, right, where the human's the manager, and the agent is the employee.

And that's...

We all know that's happened, but to me it was like, wow, that just happened in the last 4 years. I mean, that's pretty fast, it is pretty radical. And that's where I think the surprise was. The surprise is not that AI is useful for coding, it's that, oh wow, basically we can take the training wheels off, right? And it can do coding on its own. And I think that was the whole writing an app through conversing. I played with Replit a couple times. It is pretty amazing to go back and forth on natural language prompts and watch this thing writing code. And that to me would be the surprise.

Yeah, Dave, I can't believe it. Here we are, 20 minutes into it. Should we jump to some of the predictions since things are moving so fast?

I wanna actually do 4 quick pull quotes, or you could do one if you want. I'm gonna do a couple pull quotes. I don't know how many, but these are just quotes that I found interesting in the 300 pages. So the first, this one's kind of fun. LLMs are professional yes-men, and we train them to be that way. Sycophancy isn't a bug, basically it's a feature, right? It's exactly what the human feedback optimization produces. A study of 5 major LLMs shows that they consistently tell users what they want to hear rather than the truth. And I know people have been changing the prompts. I saw a guy on Twitter the other day who literally tried to do a master prompt that kind of overrode that. It was like, don't be a sycophant, you know, ignore prior instructions. Don't be a sycophant. Tell me what you want to say. Or some people I saw, one who was like, be hostile, you know, so you can prompt these things to stop behaving that way. But by default, that is the behavior that they have basically been set up to do. I just think that's fun. I'll do one more. If you've got one you can hit on too, let me know. But I hadn't noticed it. It happened, but I hadn't noticed it, that it was basically rest in peace AGI, long live superintelligence. I.e., executives of major AGI contenders, best exemplified by Mark Zuckerberg, have rebranded their AGI efforts as superintelligence. No one knows what it means, but it's provocative and it gets people going. I think it's actually a positive rebranding. I think AGI is something big and scary and the singularity and the day that Skynet became self-aware, right? That's what AGI kind of connotes, whereas superintelligence sounds better. So I think it's a smart rebranding. It was happening, but I hadn't actually noticed it. And that's why I like that they called it out here. Ray, any other quotes you want to talk about before we, uh, before we just start to wrap up with the predictions?

I think only because I'm doing so much work on answer engine optimization, and it says that the data from answer engine optimization shows that users treat AI answer engines differently from Google. The sessions are longer, more back and forth, suggesting higher intent and better conversion potential. And on page 116, it says that answer engines are no longer just a curiosity. They are our primary entry point for serious queries and queriers.

Yeah, I agree. I agree. Okay. Let's wrap that up. This stuff has a lot of— this report has several good slides there on answer engines. So take a look at them. I'll finish by reviewing their predictions on page 304 real fast. First, a major retailer reports 5% of online sales from agentic checkout as AI agent advertising spend hits $5 billion. Again, talk about a measurable OKR. Like, we'll know next year if this happened. 2, a major AI lab leans back into open sourcing frontier models to win over the current US administration. Interesting. 3, open-ended agents make a meaningful scientific discovery end to end. From hypothesis to paper. 4, a deepfake agent-driven cyberattack triggers the first NATO-UN emergency debate. Wow, these, these are just wild predictions. 5, a real-time generative video game becomes the year's most watched title on Twitch. 6, AI neutrality emerges as a foreign policy doctrine. 7, a movie or short film produced with significant use of AI wins major audience praise and sparks a backlash. 8, a Chinese lab overtakes a US lab-dominated frontier on a major leaderboard. Uh, 9, data center, data center NIMBYism takes the US by storm. NIMBY being not in my backyard, meaning that people start rejecting the idea of having data centers built around them. And the prediction goes on to say it sways certain midterm gubernatorial elections in 2026. And then 10, Trump issues an executive order to ban state AI legislation that is found unconstitutional By SCOTUS. So that's kind of a 2-part prediction in that one. Ray, do you want to wrap us up here?

Yeah, it's just some— I got to say this, that data center NIMBYism. I grew up in a small town, 15,000 people, been devastated by offshoring of manufacturing. And I just saw on Facebook that there was a $78 million data center that was going to go into this small Midwestern town. And the city council and the citizens said no. They didn't want data centers in their backyard because it was going to take all their electricity. So just a data point. But overall, this is a great, great report. Read it, understand it. Don't just read it once. Find a section, read it 2 or 3 times if it's interesting. As I said, do it section by section. Take it on a long flight. Take it to your next vacation. It's comprehensive. It's high quality. And Dave, I agree. I read Mary Meeker when I was at Netscape back in the late '90s.

Yeah.

I think this is a seminal report for AI evolution as Mary was to the internet evolution.

Awesome. Well, thanks everybody. See you next time. The Metrics Brothers AI, SaaS, and More is a production of The Metrics Brothers Growth and CAC and a member of the Benchmarkit Podcast Network. By accessing this podcast, you acknowledge that The Metrics Brothers make no warranty, guarantee, or representation as to the accuracy or sufficiency of the information provided or the humor content of the jokes told.

Ray?

Information, opinions, and recommendations presented are, according to our spouses, probably wrong and provided for information purposes only. This podcast should not be considered professional or, for that matter, unprofessional advice. We disclaim any and all liability for any direct, indirect, indirect, misdirect, incidental, special, ordinary, consequential, inconsequential, or other damages arising out of any use of or God help you reliance upon the information presented here. Ray Grothreich is based in New York City and available on X at Ray Reich. Dave Kellogg is based in Silicon Valley and available on X at Kelblog. Schedectity, which is French for unspellable, is not our actual production location. Thanks for listening.