Cursor Just Showed How Much You're Wasting On AI

Cursor rebuilt SQLite 6 different ways, and the costs ranged from $1,339 to over $10,000 on the same exact task. Today I have a guest and co-host of the pod, Kamal Hussain, who has been in the AI industry as a pioneer for 20 years plus on ML before it was cool at GitHub, Airbnb, and early LM research, doing work in that space. Hamel has worked on open source ML tools, debugging systems. He teaches the most famous AI evals course, thousands of students served and hundreds of companies, including OpenAI. And today I have him on the pod and we're going to go into technical deep dives for the entirety of the pod. And we're probably going to have a few episodes that spawn from this. I'm with Hamel at his home and we decided to just content max this whole session that we're together. So what I'm sharing on my screen here is Agent Swarms and the New Model Economics from Cursor. They rebuilt SQLite 6 different ways. This was research led by Wilson Lin on July 20th, on, in 2026. So just a couple of months ago, but I find that this article still says a lot, even in today's world. And we've been using this meta internally for my own company, Clarity. They ran an agent swarm to implement SQLite from scratch. And it was graded by this SQL Logic Test Suite. This is the Cursor blog article about it. And one of the cool things was this cost to rebuild SQLite by model mix. So you could see Opus 4.8 running tests again on Opus 5.5 as of this podcast recording would be interesting, but Opus 4.8 as the sort of brain of the operation and then Composer 2.5 ended up building SQLite at $1,339. And when you look at the very right on the spectrum in terms of cost and efficacy. We have Fable 5 just YOLOing it at about $20,000. And right before that, GPT-5.5 at $10,000. So the architecture here was really frontier model plans, worker agents execute. And the frontier model decomposes the goals in these autonomous goal loops. And on the Fable 5 config, the old swarm wrote about 64,000 lines. And when they introduced this agent swarming meta, the new swarm wrote 9,900 lines for the same task, which is really interesting because I know Hamel and I have talked about how there's a lot of verbosity and slop that comes with AI coding agents. And that's like a tension point where in a clean architecture, it's not like you want more code all the time. You do want to slim down your code, make it more elegant and refactor. So, Hamel, thoughts on this Agent Swarm Meta from Cursor proving that you can get more than 10x cost optimization for value for this, you know, particular benchmark of let's rebuild SQLite?

Yeah, I mean, one thing I, you know, I hope that happens soon is we move away from human orchestration of like, hey, like I'm going to use a strong model to do the planning and then a cheaper model to do the work or whatever. Because it's just a really bad user experience to like do the routing in your head. And so that's one thing to kind of think about. Like if you're just, yeah, I mean, you're, if you're lazy or you don't have time, you know, you are like the person on the right using Table 5 all the way. Cause you're like, I don't know. I like, I'm not going to sit here and orchestrate this. So there is some human complexity element here. Composer 2.5 is really good. It's fine-tuned. I believe it was like fine-tuned Kime model from, from Cursor. And it's really fast. It's really good if you're using Cursor. Um, again, this makes sense, except I would love to see some kind of model routing baked in. Like, could you do this with Composer 2.5 all the way? I would like to see that as a baseline. Like what happens there? I'm really curious.

Yeah, that's interesting, which is one, some, something I just thought about in your reflection on this, Hamel. As an AI builder, an ML builder for over 20 years now, and a very well-known educator for how do you objectively and measurably improve agents. And you've taught people at OpenAI. Some of the things that I think about is, you know, what questions should I ask as a result of learning this new information? So for us, like in Clarity, my team, we ship and we've created reusable skills around this meta of agent swarming, which doesn't look like it's going to go away. And so you just asked the question, which I think all AI builders should ask themselves when they see stuff like this and charts like this, which show like, oh my gosh, it's more than 10x more cost efficient while getting the same value, like SQLite rebuilt. And you were like, okay, well, how would Composer 2.5 just do on its own? And I think that's a really great question to ask as you're making sense of the world as an AI builder. Looking at this, you know,

actually let's look at that graph again. So like, if I zoom in, like, okay, so is Opus the planner or the worker? The planner.

Opus is the planner here.

So it's curious, like why in this situation, why did Opus spend so much more time planning or spend more tokens planning than in the case where it was just Opus? Yeah. Like, I guess like there's a little gray bar in the 4th thing. There's a little gray bar if you, I can see it.

Yeah.

Like what the hell is that? Is this Opus 4A planning?

Yeah, I believe that's the case.

And if that's the case, why was it Why did it spend so much less planning in that situation? Like what's going on? It's a little weird. Maybe it's just an artifact of the visual, but I don't, I think there's some, yeah, it's not zero. It looks like there is something going on.

Yeah. So it says in the run where Opus 4.8 did the planning and Composer 2.5 did the work, the entire worker fleet costs 411. Uh, and I think that looks here so that we can assume that the, oh, here's the actual token buy. model, planner versus worker. So that the other view, just going up a little bit, this view was about the cost. And then down here, this view is about the token usage. So this is interesting to drill into deeper. This is basically saying that in this agent swarm meta, if you use Opus 4.8 and Composer 2.5 in their test, they had Composer as the worker here, and it makes up a larger volume of the tokens. And that's the case across all of these. But if you look at this Opus 4.8 run, which, and this Grok 4.5 run, there's such little time spent in planning. That's the thing that jumps out here is the amount of time spent planning. So one has to wonder what prompts are they using and any reusable skills or anything like that when they run these tests, because you can get a lot of context bloat if you're nudging it sort of the wrong way. What do you think about that?

Yeah, I mean, it was like, that's one thing that stands out is like some of these more expensive ones are not spending that much time planning. The GPT-5.5 one seems like it is. It looks like from the graph I'm interpreting as using the same model for the planning and the worker, which seems fine. But yeah, like that example with the Opus 4.8, it has an asterisk on it. It says informal run for cost calibration, not part of the control comparison. Okay, so maybe there's I'm not really sure what that means.

I bet that just means we nudged it along differently and it wasn't like a true test, 'cause they're not saying it's a control. So let's read this. Few moments in a large—

It's weird to say it's not a control, but it's on the chart as a comparison.

Yeah.

So like if it's not a control, then why are we comparing this with other things?

Informal run for cost calibration. So I guess you have to read this with an asterisk, which they do put in there, but it could be drastically lower/better if this was more of a formal run for controlled comparison. Let's read a little bit about that, what they say here. Few moments in a large task genuinely require frontier intelligence, such as the original decomposition, the design decisions, and certain trade-offs. Once a frontier planner has collapsed the ambiguity into a detailed explicit instruction, less expensive models simply have to follow it. This is a huge potential source of cost savings. In the run that used GPT-5.5 for both planners and workers, the workers alone cost $9,373. In the run where Opus 4.8 did the planning, we just read this, and Composer 2.5 did the work, the entire worker fleet cost $411. One detail worth noting comes from the comparing of the 2 hybrid runs. The Fable 5 planner ran up against a slightly smaller bill than the Opus 4.8 planner, despite roughly twice the per-token price, because it used far fewer planning tokens. But the Fable's run workers went through several times as many tokens, and the run as a whole came came out substantially more expensive. That's really interesting that the Fable 5 planner put up a smaller bill than the Opus 4.8 planner, but its plans seemed to be garbage because then the workers had to run more loops is what I'm looking at this run, went through several times as many tokens and the whole run came out to be substantially—

Let's look at the graph again. So it's saying like—

Right here.

Yeah. I think it's looking at the last one here. Like, uh—

No, no, I think it's saying The hybrid runs. So in this case, Fable 5 plus Composer 2.5 versus Opus 4.8 plus Composer 2.5. And so you could see you came out with 6.7 billion tokens used. And the workers, it's saying from the plan Fable 5 came up with, the workers just ate up a lot more cost and compute. That's interesting. And in this case, they're saying that this is the controlled comparison because it doesn't have an asterisk. So it's really curious to see this huge difference.

I mean, the thing that you should take away from this article really is like, Don't use the frontier intelligence on everything.

Yeah.

It just sounds very commonsensical when you hear it come outta my mouth, but like a lot of people are not doing that. You're just getting into your coding agent and putting it on the highest mode possible and you're just going about your day.

Token maxing.

Yeah. And so I think, yeah, this just shows like, hey, there's a lot of money and time you can save by like using the right thing for the right task.

Yeah. So for the AI builders out there, Hamel, could you summarize again, like some of your thoughts of this token economics, like agent swarm economics article from Cursor? Like what should AI builders be thinking about when it comes to maximizing the value of their tokens with this agent swarm method?

Yeah, I mean, it looks like you should use the right model for the task. Like there's just like the article says, like you don't need frontier intelligence for everything. So don't use it for everything because frontier intelligence is like significantly more expensive. Than just like one level down. If you go like a few levels down in intelligence, now all of a sudden it's like, I don't know, like 10, 20 times cheaper. So yeah, I mean, I think we already knew that, but it is good to quantify this. It would be good if the UX of doing this was better. I think there is some things that you can use that's like, I haven't used Cursor in a little bit. It could be like auto mode. And I'm curious if this is what auto mode does. like does routing for you. That's something that maybe we should find out real quick.

Okay, let's look at this. So Cursor Router auto mode is now powered by Cursor Router as of July 22nd, 2026. Cursor Router is our intelligent model router.

There you go.

Analyzes each request and sends it to the right model for the job. Frontier models handle work that demands them. Price-efficient models handle the rest. Let's take a look at this quick video. So we see there's optimization modes. You could edit for intelligence, balance, and cost. You can ask it a simple query, create a new PR for UI updates. And if you're using cursor auto mode, then it will optimize how the router should be used towards whatever goal. So intelligence frontier quality matching the most expensive and powerful models that might be out of reach for daily use. Balance strong quality matching the frontier models that most people like to daily drive. And cost good quality reaching the highest available intelligence while optimizing token spend. Balance and intelligence bill at the routed model's rate. Each mode moves you along the cost intelligence Pareto frontier. So there's some admin controls that people could do in Cursor teams for this. Do you think this is just like commodity meta at this point for AI builders?

No, I mean, like a lot of agent harnesses don't have routing in it, as far as I know. Does Cloud Code have routing in it? Do you think like some—

I have to force it. No, like I think.

I'm not using, I'm using Codex.

I just recently—

I don't know if Codex has it either, but I've never used the most recent versions of Claude Code.

Yeah, I just switched back to Claude Code because I ran out of Codex tokens and I got a ship. So with Opus 5.5, I just started using it again. I have found that I've had to force it to use a particular skill that does this agent swarm meta. One of the things I'll say for folks is just take this article. And throw it into your coding agent of choice and say, hey, what can we learn from this? And can you build me a reusable skill to go with this planner-worker type meta? I built a skill called Brain Hands where I use a frontier intelligence model like Fable 5.1 as the brains of the operations, and then a cheaper model such as Opus 5 or 5.5 now, or GPT-5.6 as the hands. And I've had a lot of success with that, actually. I typically nowadays have really long-running autonomous goal loops, and I need to know that it's doing the right thing. So I have it create evidence documents. That's kind of my thing. Are you using any reusable skills, Hamel?

I have a lot. Yeah, I have a lot of skills. No, I'm not using any skills. I'm just kidding.

YOLO. I'm just

giving you a hard time. No, there's a lot of skills I use for my business. You know, the repeatable tasks and stuff.

Anything like this though, where it's like Frontier Intelligence planner and then, you know, more cheaper models?

Not really. Like I don't try to overcomplicate my life. Like sometimes I'll plan with, you know, a Frontier Intelligence and then I'll, I'll do something similar, but sometimes I just try with like a smaller model and I have really good success.

Okay.

I don't try to over-optimize.

Gotcha. Okay. That makes sense.

I try, I pay attention if I start running out of tokens, I start running out of like usage.

Well, I think that's exactly the case is you mainly start paying attention when you need to token optimize.

Yeah. But I've gotten really good luck by just going down to Astra Lite reasoning.

Oh, really?

Wait, say more about that. And I get almost everything done, but with Astra Lite reasoning.

Tell me more about the Astra Lite reasoning and how you use it for AI builders so they could kind of understand. Yeah.

I mean, so like you have all the different models and codecs. I think Astra is the frontier model, like the most powerful intelligence one. And they have the reasoning levels like light, medium, high, max, something else beyond that. I think like previously, if you're using these models, like using your coding harnesses like a year ago, you would, it was really common to just set everything the highest.

Yeah.

Let me get the best model with the highest reasoning level.

All the intelligence. Keep going. Think a lot.

Now with Astra, like it's very expensive, so you can run out of your quota really fast. And I started running out of my quota all of a sudden. I think a lot of people have that experience with Codex. You said you had that experience with

Codex? I had that experience. I just ran out of tokens today.

And so I was like, you know, I probably don't need all of this. Like, I was just being lazy. So I was like, okay, like as a first step before going off of Astra, let me just set it to light reasoning and see what happens and

see how it does for

you. Most of the tasks that I needed to do, they were great. You can see even here on this chart, right?

Like, Here's Frontier Math.

Yeah, it seems like there's a plateau after a certain—

Yeah, here's Terminal Bench Science. Terminal Bench 4.0. Yeah. So we could see here on OpenAI's blog here introducing GPT-6 Astra for the AI builders out there that are looking to token optimize. Looking at Terminal Bench 4.0 as a benchmark in terms of, let's put it as output tokens, we could see over here, GPT-6 Astra reasoning effort max, You get 56.7% accuracy and 72,000 output tokens.

See if they have a benchmark with GPT-6.1 SOL, which just came out today. Do you think there might be something like that?

It would be cool. I don't know. Here, let's see if this comes out. Oh yeah, here we go. For all the AI builders out there, GPT-6.1 SOL just came out. You get near Astra Intelligence, their most capable frontier model, for a fifth of the price. OpenAI blog says, we're introducing GPT-6.1 SOL, an upgrade to GPT-6 SOL, that nearly matches GPT-6 Astra's intelligence on agentic coding, computer use, and professional work at 1/5 of Astra's standard input and output token prices. Cached input costs just $0.10 per million tokens, 95% less than standard input pricing and 50% less than GPT-6 Sol's cached input pricing, giving developers more room to build and run capable agents that reuse context across requests. And here are some cards that kind of talk about it, uh, near Astra Intelligence.

There you go. And so like, this is a good reason to just use 6.1 SOL and like at least try it on your workflows. 'Cause like, yeah, it's way cheaper than even Astra.

Totally. So Hamel, your point for AI builders out there that are trying to token optimize is really like, I mean, you even see it here, which is, hey, maybe don't like over-optimize your own individual harness or skills. But at the, as a first step to keep your life simple, because all these things are changing all the time. It's hard to stay on top of the meta and the latest on X or Twitter or GitHub. Yeah.

It's like, basically you don't need Frontier. You might not need Frontier Intelligence for anything.

Yeah. Maybe you don't.

Like, if you look at this, it might take a little bit of an ego.

Yeah.

Like a hit to your ego.

Yeah.

But like, maybe you don't.

If you look at it as well, GPT 6.1, sole effort max, cost per task, $1.57, score 71.9. On DeepSuite, which I think is one of the most comprehensive, close to real world.

Yeah. And if you see on here, like, so like this GPT-6.1 SOL is Frontier Intelligence even more. It's more powerful than Astra. I

think it levels out. Like if we look at the highest, it's nearly as powerful. Cost per task, $1.57. On the max for Astra, it's about 5x that. So $7.50 for only 1. 3 points more on DeepSuite.

Yeah, when I'm looking at this, like in this benchmark, like the second reasoning, highest reasoning level seems like a really sweet spot. What is that, medium?

Also, even looking, if you drop it from max and extra high, it scores higher. So it

might be, it might be trying to overcook things.

From your experience as an AI builder, what do you mean by overcooking things? What do you mean by that?

Well, sometimes when we turn the reasoning level up too high, it just overthinks. It literally overthinks what it's doing.

Literally overthinking.

Because that's what you're telling it to do is just think more, think more. Things don't need that much thinking as you think it does.

Might be overcooking.

And sometimes it can hurt performance. And that's what it looks like happened here.

So what's your advice to AI builders that are trying to stay on top of the latest meta and then make use of some of this like model routing, token optimization? Would you say like it's, hey, just Try the latest model and try the different reasoning efforts and then see on real world.

Yeah, try the latest model, try it at a low reasoning level and be conservative. And that's going to save you a lot right there. And that might just solve your problem. And then like, who cares? You don't need all the like model routing and skills or whatever. Just keep your life simple. You might have to go there at an enterprise level. I think model routing does seem like it makes sense.

Mm-hmm.

Because you can amortize the cost of like model routing across a lot of people. You can tune things and whatnot. You know, the Cursor thing you just went over seems great too about like their model router. So, I mean, I would just try all these things. A year ago I was biased towards like, okay, you need to go for the frontier intelligence on everything. Now it's like, you still might want the frontier intelligence, but like, they're so good now that you might not need it. Like, let's just be real.

Definitely.

Like, what are you making?

Yeah.

Do you really need it?

Are you creating net new scientific research in the world? You

know? Yeah. I mean, whatever it is, like, it's worth trying cheaper models because they're just good and they're so much cheaper. So like, do that. And it's going to be— it's going to run faster too. You're going to get the task done faster because smaller models are faster.

Thanks, Kamal, for joining the pod. And for everybody listening, hit subscribe wherever you're at— Spotify, Apple Podcasts, YouTube. And follow Hamel. I'll have links to his, all of his stuff and his content in the show notes. And please take his course if you're an AI builder trying to increase your economic value with your skills in this AI landscape, which is changing all the time. But the thing that isn't going to change is, can you objectively and measurably improve your LLM-based products? That's not going to change. So these skills are cool and a bunch of hard-earned secrets. You could take his class and learn just like I did. So anyway, thanks, Emil, for being an awesome guest.

Thanks for having me. Thank you.

Have a great day. Take care, AI builders. Bye-bye.