Cursor Team: Future of Programming with AI | Lex Fridman Podcast #447

The following is a conversation with the founding members of the Cursor team, Michael Trul, Swale, Asif Arvid Lunmark and Aman Sanger. Cursor is a code editor based on VS Code that adds a lot of powerful features for AI assisted coding. It has captivated the attention and excitement of the programming and AI communities. So I thought this is an excellent opportunity to dive deep into the role of AI in programming. This is a super technical conversation that is bigger than just about one code editor. It's about the future of programming and in general, the future of human AI collaboration in designing and engineering complicated and powerful systems. This is the Lex Friedman podcast to support it. Please check out our sponsors in the description. And now, dear friends, here's Michael Suale, Arvid and Aman. All right, this is awesome. We have Michael, Aman, Suale, Arvid here from the Cursor team. First up, big ridiculous question. What's the point of a code editor?

So the code editor is largely the place where you build software. And today, or for a long time, that's meant the place where you text edit a formal programming language. And for people who aren't programmers, the way to think of a code editor is like a really souped up word processor for programmers where the reason it's souped up is code has a lot of structure and so the word processor, the code editor, can actually do a lot for you that word processors sort of in the writing space haven't been able to do for people editing text there. And so that's everything from giving you visual differentiation of the actual tokens in the code so you can scan it quickly to letting you navigate around the code base. Sort of like you're navigating around the Internet with hyperlinks. You're going to definitions of things you're using to error checking to catch rudimentary bugs. And so traditionally that's what a code editor has meant. And I think that what a code editor is, is going to change a lot over the next 10 years as what it means to build software maybe starts to look a bit different.

I think also a code editor should just be fun.

Yes, that is very important. That is very important. And it's actually sort of an underrated aspect of how we decide what to build. Like a lot of the things that we build and then we try them out, we do an experiment and then we actually throw them out because they're not fun. And so a big part of being fun is being fast. A lot of the time fast is fun.

Yeah. Fast. Yeah, that should be a T shirt.

Fundamentally, I think one of the things that draws a lot of people to building stuff on computers is this insane iteration speed where in other disciplines you might be gatekept by resources or even the ability to get a large group together. And coding is this amazing thing where it's you and the computer and that alone you can build really cool stuff really quickly.

So for people who don't know, Cursor is this super cool new editor that's a fork of VS code. It would be interesting to get your kind of explanation of your own journey of editors. How did you, I think all of you were big fans of VS code with Copilot. How did you arrive to VS code? And how did that lead to your journey with Cursor?

Yeah, so I think a lot of us, well, all of us were originally VIM users.

Pure, Pure vim.

Pure vim. Yeah. No neovim, just pure Vim in a terminal. And at least for myself, it was around the time that copilot came out, so 2021, that I really wanted to try it. So I went into VSCode, the only platform, the only code editor in which it was available. And even though I really enjoyed using Vim, just the experience of Copilot with VS code was more than good enough to convince me to switch. And so that kind of was the default until we started working on Cursor.

And maybe we should explain what Copilot does. It's like a really nice autocomplete. It suggests, as you start writing a thing, it suggests one or two or three lines how to complete the thing. And there's a fun experience in that, you know, like when you have a close friendship and your friend completes your sentences, like when it's done well, there's an intimate feeling. There's probably a better word than intimate, but there's a. There's a cool feeling of like, holy shit, it gets me. And then there's a. An unpleasant feeling when it doesn't get you. And so there's that kind of friction. But I would say for a lot of people, the feeling that it gets me overpowers that it doesn't.

And I think actually one of the underrated aspects of GitHub Copilot is that even when it's wrong, it's like a little bit annoying. But it's not that bad because you just type another character and then maybe then it gets you, or you type another character and then it gets you. So even when it's wrong, it's not that bad?

Yeah, you can sort of iterate and fix it. I mean, the other underrated part of Copilot for me sort of was just the first real AI product. It's like the first language model consumer product.

So Copilot was kind of like the first killer app for LLMs.

And like the beta was out in 2021.

Right. Okay, so what's the origin story of Cursor?

So around 2020, the scaling loss papers came out from OpenAI and that was a moment where this looked like clear, predictable progress for the field, where even if we didn't have any more ideas, it looks like you could make these models a lot better if you had more compute and more data.

By the way, we'll probably talk for three to four hours on the topic of scaling loss. Just to summarize, it's a paper and a set of papers and a set of ideas that say bigger might be better for model size and data size. In the realm of machine learning, it's.

Bigger and better, but predictably better.

Okay, that's another topic of conversation.

But anyway, so around that time for some of us, there were a lot of conceptual conversations about what's this going to look like, what's the story going to be for all these different knowledge worker fields about how they're going to be made better by this technology getting better. And then I think there were a couple of moments where the theoretical gains predicted in that paper started to feel really concrete. And it started to feel like a moment where you could actually go and not do a PhD if you wanted to work on, do useful work in AI. Actually felt like now there was this whole set of systems one could build that were really useful. And I think that the first moment we already talked about a little bit, which was playing with the early beta of copilot. That was awesome and magical. I think that the next big moment where everything kind of clicked together was actually getting early access to GPT4. So sort of end of 2022 was when we were tinkering with that model and the step up in capabilities felt enormous. And previous to that, we had been working on a couple of different projects because of copilot, because of scaling oz, because of our prior interest in the technology, we had been tinkering around with tools for programmers, but things that are very specific. So we were building tools for financial professionals who have to work within a jupyter notebook or playing around with can you do static analysis with these models? And then the step up in GPT4 felt like, look, that really made concrete the theoretical gains that we had predicted before. Felt like you could build a lot more just immediately at that point in time. And also if we were being consistent, it really felt like this wasn't just going to be a point solution thing. This was going to be all of programming was going to flow through these models. It felt like that demanded a different type of programming environment, a different type of programming. And so we set off to build that sort of larger vision around that.

There's one that I distinctly remember. So my roommate is an IMO gold winner. And there's a competition in the US called the Putnam, which is sort of the IMO for college people. And it's this math competition is exceptionally good. So Sheng Tang and Amon, I remember it's sort of June of 2022, had this bet on whether the mod, like 2024, June or July, you were going to win a gold medal in the IMO with like models.

IMO is International Math Olympiad.

Yeah, IMO is International Math Olympiad. And so Arvid and I are both also competed in it. So it was sort of personal. I remember thinking, matt, this is not going to happen. This was like, even though I sort of believed in progress, I thought, you know, I'm a girl just like Ahmad is, just delusional. That was the. And to be honest, I mean I was, to be clear, very wrong. But that was maybe the most prescient bet in the group.

So the new results from DeepMind, it turned out that you were correct. That's what this.

Well, it was technically.

Not technically incorrect, but one point away.

Iman was very enthusiastic about that stuff. And before Iman had this scaling laws T shirt that he would walk around with where it had the charts and the formulas on it.

So you felt the AGI or you felt the scaling laws.

Yeah, I distinctly remember there was this one conversation I had with Michael where before it hadn't thought super deeply and critically about scaling laws. And he kind of posed the question, why isn't scaling all you need? Or why isn't scaling going to result in massive gains in progress? And I think I went through the stages of grief. There is anger, denial, and then finally at the end, just thinking about it, acceptance. And I think I've been quite hopeful and optimistic about progress since I think one thing I'll caveat is I think it also depends on which domains you're going to see progress. Math is a great domain because. Especially formal theorem proving because you get this fantastic signal of actually verifying if the thing was correct and so this means something like RL can work really, really well. And I think like you could have systems that are perhaps very superhuman at math and still not technically have AGI.

Okay, so can we take it all the way to cursor? And what is cursor? It's a fork of VS code. And VS code is one of the most popular editors. For a long time, like everybody fell in love with it, Everybody loved vim. I left Emacs for it. Sorry. So unified in some fundamental way, the developer community. And then you look at the space of things, you look at the scaling laws. AI is becoming amazing. And you decided, okay, it's not enough to just write an extension for your VS code because there's a lot of limitations to that. We need, if AI is going to keep getting better and better and better, we need to really like rethink how the AI is going to be part of the editing process. And so you decided to fork VS code and start to build a lot of the amazing features we'll be able to talk about. But what was that decision like? Because there's a lot of extensions, including copilot of VS code that are doing sort of AI type stuff. What was the decision like to just fork VS code?

So the decision to do an editor seemed kind of self evident to us for at least what we wanted to do and achieve. Because when we started working on the editor, the idea was these models are going to get much better, their capabilities are going to improve and it's going to entirely change how you build software, both in a, you will have big productivity gains, but also radical in how the act of building software is going to change a lot. And so you're very limited in the control you have over a code editor if you're a plugin to an existing coding environment. And we didn't want to get locked in by those limitations. We wanted to be able to just build the most useful stuff.

Okay, well then the natural question is, you know, VS code is kind of with Copilot, a competitor, so how do you win? Is it basically just the speed and the quality of the features?

Yeah, I mean, I think this is a space that is quite interesting, perhaps quite unique, where if you look at previous tech waves, maybe there's kind of one major thing that happened and it unlocked a new wave of companies. But every single year, every single model capability or jump you get in model capabilities, you now unlock this new wave of features, things that are possible especially in programming. And so I think in AI programming being even just a few Months ahead, let alone a year ahead, makes your product much, much, much more useful. I think the cursor a year from now will need to make the cursor of today look obsolete. And I think Microsoft has done a number of fantastic things, but I don't think they're in a great place to really keep innovating and pushing on this in the way that a startup can.

Just rapidly implementing features.

Yeah. And kind of doing the research, experimentation necessary to really push the ceiling.

I don't know if I think of it in terms of features as I think of it in terms of capabilities for programmers. It's that as the new Zero1 model came out, and I'm sure there are going to be more models of different types, like longer context and maybe faster, there's all these crazy ideas that you can try and hopefully 10% of the crazy ideas will make it into something kind of cool and useful. And we want people to have that sooner. To rephrase, it's like an underrated fact is we're making it for ourselves. When we started Cursor, you really felt this frustration that you could see models getting better, but the COBOL experience had not changed. It was like, man, this, These guys, like, the ceiling is getting higher. Like, why are they not making new things? Like, they should be making new things. They should be like, where's all the alpha features? There were no alpha features. I'm sure it was selling well, I'm sure it was a great business, but it didn't feel. I'm one of these people that really want to try and use new things and there's no new thing for a very long while.

Yeah, it's interesting. I don't know how you put that into words, but when you compare cursor with copilot, Copilot pretty quickly started to feel stale for some reason.

Yeah, I think one thing that I think helps us is that we're sort of doing it all in one, where we're developing the UX and the way you interact with the model at the same time as we're developing how we actually make the model give better answers. So how you build up the prompt or how do you find the context and for a cursor tab, like, how do you train the model? So I think that helps us to have all of it sort of like the same people working on the entire experience end to end.

Yeah, it's like the person making the UI and the person training the model sit 18ft away.

So often the same person even.

Yeah, often. Even the Same person. You can create things that are sort of not possible if you're not talking, you're not experimenting and you're using, like.

You said, cursor to write Cursor, of course.

Oh, yeah, yeah.

Well, let's talk about some of these features. Let's talk about the all knowing, the all powerful. Praise be to the tab. You know, autocomplete on steroids, basically. So what, how does tab work?

What is tab to highlight and summarize at a high level? I'd say that there are two things that cursor is pretty good at right now. There are other things that it does, but two things it helps programmers with. One is this idea of looking over your shoulder and being a really fast colleague who can kind of jump ahead of you and type and figure out what you're going to do next. And that was the original idea behind that was kind of the kernel of the idea behind good autocomplete was predicting what you're going to do next. But you can make that concept even more ambitious by not just predicting the characters after your cursor, but actually predicting the next entire change you're going to make, the next diff, the next place you're going to jump to. And the second thing cursor is pretty good at right now too, is helping you sometimes jump ahead of the AI and tell it what to do and go from instructions to code. And on both of those, we've done a lot of work on making the editing experience for those things ergonomic and also making those things smart and fast.

One of the things we really wanted was we wanted the model to be able to edit code for us. That was kind of a wish and we had multiple attempts at it before we had a good model that could edit code for you. Then after we had a good model, I think there have been a lot of effort to make the inference fast for having a good experience. And we've been starting to incorporate, I mean, Michael sort of mentioned this like ability to jump to different places. And that jump to different places I think came from a feeling of, you know, once you accept an edit, it was like, man, it should be just really obvious where to go next. It's like I'd made this change. The model should just know that like the next place to go to is like 18 lines down. Like if you're a WIM user, you could press 1, 8 JJ or whatever. But why am I doing this? The model should just know it. And then so the idea was you just press tab, it would go 18 lines down and then make it show you the next edit. And you would press tab. So it's just as long as you could keep pressing tab. And so the internal competition was how many tabs can we make the one press? Once you have the idea more sort of abstractly, the thing to think about is sort of like how are the edits sort of zero entropy. So once you've sort of expressed your intent and the edit is there's no new bits of information to finish your thought, but you still have to type some characters to make the computer understand what you're actually thinking, then maybe the model should just read your mind and all the zero entropy bits should just be tabbed away. Yeah, that was sort of the abstract version.

There's this interesting thing where if you look at language model loss on different domains, I believe the bits per byte, which is kind of character normalized loss for code is lower than language. Which means in general there are a lot of tokens in code that are super predictable, a lot of characters that are super predictable. And this is I think even magnified when you're not just trying to autocomplete code, but predicting what the user is going to do next in their editing of existing code. And so the goal of cursor tap is let's eliminate all the low entropy actions you take inside of the editor. When the intent is effectively determined. Let's just jump you forward in time, skip you forward.

Well, what's the intuition and what's the technical details of how to do next cursor prediction? That jump that's not so intuitive, I think to people.

Yeah, I think I can speak to a few of the details on how to make these things work. They're incredibly low latency, so you need to train small models on this task. In particular, they're incredibly pre fill token hungry. What that means is they have these really, really long prompts where they see a lot of your code and they're not actually generating that many tokens. And so the perfect fit for that is using a sparse model, meaning an MOE model. So that was kind of one breakthrough we made that substantially improved its performance at longer context. The other being a variant of speculative decoding that we kind of built out called speculative edits. These are two, I think important pieces of what it quite high quality and very fast.

Okay, so MOE mixture of experts. The input is huge, the output is small.

Yeah.

Okay, so what else can you say about how to make is caching play.

A role in this particular. Caching plays a huge role because you're dealing with this many input tokens, if every single keystroke that you're typing in a given line, you had to rerun the model on all of those tokens passed in, you're just going to one, significantly degrade latency. Two, you're going to kill your GPUs with load. So you need to design the actual prompts used for the model such that they're caching aware. And then, yeah, you need to reuse the KV cache across requests just so that you're spending less, work less compute.

Again, what are the things that tab is supposed to be able to do kind of in the near term just to sort of linger on that generate code like fill empty space. Also edit code across multiple lines and then jump to different locations inside the.

Same file and then like jump to different files also. So if you make an edit in one file and maybe you have to go to another file to finish your thought, it should go to the second file also.

Yeah, and then the full generalization is like next action prediction. Sometimes you need to run a command in the terminal and it should be able to suggest the command based on the code that you wrote too. Or sometimes you actually need to suggest something, but it's hard for you to know if it's correct because you actually need some more information to learn. You need to know the type to be able to verify that it's correct. And so maybe it should actually take you to a place that's like the definition of something and then take you back so that you have all the requisite knowledge to be able to accept the next completion.

So providing the human the knowledge.

Yes.

Right.

Yeah.

Can you integrate like, I just gotten to know a guy named primegen who I believe has an ssh. You can order coffee via ssh.

Oh yeah.

Oh, we did that.

We did that.

So can that also the model do that, like feed you and like, and provide you with caffeine? Okay, so that's the general framework.

Yeah.

And the magic moment would be if programming is this weird discipline where sometimes the next five minutes, not always, but sometimes the next five minutes of what you're going to do is actually predictable from the stuff you've done recently. And so can you get to a world where that next five minutes either happens by you disengaging and it taking you through, or maybe a little bit more of just you seeing next step, what it's going to do, and you're like, okay, that's good, that's good, that's good. That's good. And you can just sort of tap, tap, tap through these big changes.

As we're talking about this, I should mention like, one of the really cool and noticeable things about Cursor is that there's this whole diff interface situation going on. So like the model suggests with the red and the green of like, here's how we're going to modify the code. And in the chat window you can apply and it shows you the diff and you can accept the diff. So maybe can you speak to whatever direction of that?

We'll probably have like four or five different kinds of diffs. So we have optimized the diff for the autocomplete. So that has a different diff interface when you're reviewing larger blocks of code. And then we're trying to optimize another diff thing for when you're doing multiple different files and sort of at a high level. The difference is for when you're doing autocomplete, it should be really, really fast to read. Actually it should be really fast to read in all situations. But in autocomplete it's sort of, you're really like your eyes focused on one area. You can't be in too many. The humans can't look in too many different places.

So you're talking about on the interface side.

On the interface side. So it currently has this box on the side. So we have the current box. And if it tries to delete code in some place and tries to add other code, it tries to show you.

A box on the side, you can maybe show it. If we pull it up on cursor.com this is what we're talking about.

So that box, it was like three or four different attempts at trying to make this thing work. Where first the attempt was like these blue crossed out lines. So before it was a box on the side, it used to show you the code you delete by showing you like Google Doc style. You would see like a line through it. Then you would see the new code that was super distracting. And then we tried many different, you know, there was sort of deletions, there was trying to red highlight. Then the next iteration of it, which is sort of funny, would you would hold the, On Mac, the option button. So it would sort of highlight a region of code to show you that there might be something coming. So maybe in this example, like the input and the value would all get blue and the blue would highlight that the AI had a suggestion for you. So instead of directly showing you the thing it would show you that the AI, it would just hint that the AI had a suggestion. And if you really wanted to see it, you would hold the option button and then you would see the new suggestion. And if you release the option button, you would then see your original code.

So that's. By the way, that's pretty nice. But you have to know to hold the option button.

Yeah.

By the way, I'm not a Mac user, but I got it. It's a button. I guess you people have.

Again, it's just non intuitive. I think that's the key thing.

And there's a chance this is also not the final version of it.

I am personally very excited for making a lot of improvements in this area. We often talk about it as the verification problem, where these divs are great for small edits, for large edits, or when it's multiple files or something. It's actually a little bit prohibitive to review these diffs. And so there are a couple of different ideas here. One idea that we have is, okay, parts of the diffs are important, they have a lot of information. And then parts of the diff are just very low entropy. They're the same thing over and over again. And so maybe you can highlight the important pieces and then gray out the not so important pieces. Or maybe you can have a model that looks at the diff and sees, oh, there's a likely bug here. I will mark this with a little red squiggly and say you should probably review this part of the diff and ideas in that vein I think are exciting.

Yeah, that's a really fascinating space of UX design engineering. So you're basically trying to guide the human programmer through all the things they need to read and nothing more. Like optimally.

Yeah, and you want an intelligent model to do it. Like currently diff algorithms are just like normal algorithms. There's no intelligence. There's like intelligence that went into designing the algorithm, but then there's no. Like you don't care if it's about this thing or this thing as you want a model to do this.

So I think the general question is like, Matt, these models are going to get much smarter. As the models get much smarter, the changes they will be able to propose are much bigger. So as the changes get bigger and bigger and bigger, the humans have to do more and more and more verification work. It gets more and more and more. You need to help them out. I don't want to spend all my time reviewing code.

Can you say a little more across multiple files?

Div.

Yeah, I mean, so GitHub tries to solve this, right, with code review. When you're doing code review, you're reviewing multiple diffs across multiple files. But like Arvid said earlier, I think you can do much better than code review. Code review kind of sucks. You spend a lot of time trying to grok this code that's often quite unfamiliar to you, and it often doesn't even actually catch that many bugs. I think you can significantly improve that review experience using language models, for example, using the kinds of tricks that Arvid had described of maybe pointing you towards the regions that actually matter. I think also if the code is produced by these language models and it's not produced by someone else, the code review experience is designed for both the reviewer and, and the person that produced the code. In the case where the person that produced the code is a language model, you don't have to care that much about their experience and you can design the entire thing around the reviewer such that the reviewer's job is as fun, as easy, as productive as possible. And I think that feels like the issue with just kind of naively trying to make these things look like code review. I think you can be a lot more creative and push the boundary on what's possible.

Just one idea there is, I think ordering matters. Generally when you review a pr, you have this list of files and you're reviewing them from top to bottom. But actually you actually want to understand this part first, because that came logically first. And then you want to understand the next part. And you don't want to have to figure out that yourself. You want a model to guide you through the thing.

And is the step of creation going to be more and more natural language is the goal versus with actual.

I think sometimes I don't think it's going to be the case that all of programming will be natural language. And the reason for that is, you know, if I'm pair programming with Zwala and Zwala's at the computer and the keyboard, and sometimes if I'm like driving, I want to say to swale, hey, implement this function. And that works. And then sometimes it's just so annoying to explain to Sualev what I want him to do. And so I actually take over the keyboard and I show him, I write part of the example and then it makes sense. And that's the easiest way to communicate. And so I think that's also the case for AI. Sometimes the easiest way to communicate with the AI will be to show an example and then it goes and does the thing everywhere else. Or sometimes if you're making a website, for example, the easiest way to show to the AI what you want is not to tell it what to do, but drag things around or draw things and maybe eventually we will get to brain machine interfaces or whatever and kind of understand what you're thinking. And so I think natural language will have a place. I think it will definitely not be the way most people program most of the time.

I'm really feeling the AGI with this editor. It feels like there's a lot of machine learning going on underneath. Tell me about some of the ML stuff that makes it all work.

Recursor really works via this ensemble of custom models that we've trained alongside the frontier models that are fantastic at the reasoning intense things. And so Cursortap, for example, is a great example of where you can specialize this model to be even better than even frontier models. If you look at evals on the task, we set it at the other domain, which it's kind of surprising that it requires custom models, but it's kind of necessary and works quite well is in apply. So I think these models are like the frontier models are quite good at sketching out plans for code and generating rough sketches of the change. But actually creating diffs is quite hard for frontier models. For on your training models, you try to do this with Sonnet with o any frontier model and it really messes up stupid things like counting line numbers, especially in super super large files. And so what we've done to alleviate this is we let the model kind of sketch out this rough code block that indicates what the change will be and we train a model to then apply that change to the file.

And we should say that apply is the model, looks at your code, it gives you a really damn good suggestion of what new things to do. And the seemingly for humans trivial step of combining the two, you're saying is not so trivial.

Contrary to popular perception, it is not a deterministic algorithm.

Yeah, I think you see shallow copies of apply elsewhere and it just breaks most of the time because you think you can kind of try to do some deterministic matching and then it fails at least 40% of the time and that just results.