AI Code Review Hot Takes with Merrill Lutsky, CEO at Graphite

Writing code used to be 80% of the job. Now it's maybe 10%, 20%, not even that if you have an agent who can go off and do this for you.

The role of engineering is really shifting from writing code to reviewing code. It's not enough to just bring AI to the IDE. That outer loop of development can sometimes be more of a bottleneck than, than actually writing the code. Combining those kind of deterministic and more traditional methods with agentic review and having like a code review companion that we call Diamond to review every single code change in a matter of seconds. Our unique take is that you need both of those combined. Combine all into one platform in order to properly handle the volume of code that we're seeing generated today.

Now we're sort of in the agents era, I guess you could call it background agents. You can give an asynchronous task, they'll go off and do it for you. What's next is the agent is slacking you and it's saying, hey, I've got this idea for a great feature, you know, can you please go get me a Graphite API key so I can implement it?

That becomes the place where humans and agents interact. Like the agent is handing you back a code change, it's handing you back a PR, and you are then interacting with it.

Something got added in, uh, in Claude 4 where it loves at the end of Completing a task, it'll spend about 2 paragraphs just complimenting itself and saying how good of a job it did. And we're live. Well, welcome, Miroslav Lutsky. Thank you so much for taking the time. Excited to learn more about Graphite, your efforts related to agent stacking and all of the related things you guys are working on currently. So I think a great way to get started is, would you mind giving me a brief personal introduction and also tell me what is Graphite, what is stacking, and how does this apply to agentic coding?

Yeah, happy to. And thanks for having me, Jay. It's always fun to, to have conversations with companies where there are a lot of common customers and with Graphite and a lot of folks that are kind of getting the benefits of both. So yeah, excited to, to share more about this. Yeah, brief, brief personal background. So Merrill Lutzky, co-founder and CEO of Graphite. Started my career about, I guess now 12 years ago. I did, I like took some time off of school, did a YC startup. Terrible business idea, but ended up, you know, taking a year off from school, learning a hell of a lot about like building companies. Um, had a small acquisition out of that. Wasn't successful enough to become one of the famous Harvard dropouts. Returned to school and actually ended up being a wonderful thing in that I met my co-founders, Tomas and Greg. Uh, they were a couple of years below me, but they were two of the hardest working and then most curious CS kids in the startup community. So. The three of us kind of bonded pretty quickly over that. All kind of went to big companies after school. So I was at Square for a while and then Oscar Health here in New York and a couple other startups. Tomas went to Meta for a while and then Greg was at Google and Airbnb. All kind of did our time at bigger companies, got exposure to what amazing dev toolchains look like. And then, you know, all kind of had the collective realization that what is available at companies of that scale and what they've built to support orgs where you have thousands of developers all working together and in a massive monorepo. It's just so different from what's available on GitHub and what startups have available starting out is just so far behind that internal tooling. And initially when we started what became Graphite, we were building something different in dev tools, but it was the experience of hiring some of Tomasz's ex-Meta coworkers and kind of seeing their frustration around like doing code review on GitHub and not having workflows like stack PRs or a good merge queue for handling all the merging, all these PRs in at once. That frustration was largely what, uh, what inspired us to first build the internal tool that became Graphite and then eventually to make it into a company in and of itself. Today, well, we say that Graphite is building the code review platform for this age of code gen. Uh, we think that the extent to which AI has enabled companies to ship faster, to create features faster, it's an amazing force multiplier for engineers, but it's not enough to just bring AI to the IDE or to the dev— to like the process of creating code. Anyone who's worked in a team knows that that's only the first half of the story. And you then need to also make sure that that code is reviewed, that it's tested, that it's deployed properly. And all of that, you know, that outer loop of development can sometimes be more of a bottleneck than actually writing the code, even before, and this was true even before we got these amazing, you know, these amazing agents that can help us do that. So basically that problem has become 10x harder for, you know, if we have these 10x engineering agents, that just makes the problem of code review 10x more important or 10x more painful. For companies that are using them. And we say also that we combine both like deterministic and non-deterministic methods. So taking things like stack pull requests, like a better code review interface, automations around reviewer assignment and who should take what action at what time, a merge queue to coordinate merging all these things in. But combining that, those kind of deterministic and more traditional methods with now, you know, agentic review and having like a code review companion that we call Diamond. To review every single code change in a matter of seconds. It's only, you know, there's some companies that do one, one side of that. There's some companies that do the other. Our unique take is that you need both of those combined all into one platform in order to, to properly handle the volume of code that we're seeing generated today. That's what Graphite is doing.

So I know a lot of software engineers have not been exposed to stacking previously. I personally think it's a, you know, fantastic way of rigging up a human agentic collaboration system, especially if you're trying to build code very quickly. Do you mind taking us through what that actually consists of? And then explain why it is a convenient way to arrange work between humans and agents.

Yeah, happy to. So the idea of stack pull requests, and as a concept, this existed even long before we had these ideas of agents and AI-generated code. The idea of stacking is that instead of writing, you know, let's say that you're building a feature, instead of writing that all in one massive pull request and you combine your model definitions, your API endpoints, all of your client code, all in like one monolithic PR. You put that up on GitHub, you wait, you tag a bunch of reviewers for all those different parts of the codebase, and then you wait for everybody to give you an approval. Stack pull requests, the idea is that you instead break that big change into a series of component parts where each one is independently— you take one part of that work, just your model definitions, just your API endpoints. There are many different ways to think about like breaking up a stack, but basically you want every item in the stack to be like independently reviewable and mergeable. And the beautiful part about it is that then instead of getting this, this massive monolithic PR, you get this chain of dependencies of like smaller PRs that can be really quickly reviewed. And you also unblock like building on top of the prior dependency. The problem you have with a monolithic model is that let's say that you write a bunch of code, you're then waiting for somebody to give you a review. You're then blocked of whether, you know, whether you can either like wait for that person to get back to you with feedback, or you can just keep pushing more changes and make the PR even more complicated and even harder to review. So this model was, you know, was developed at Meta and Google developed a similar thing. Their challenge was, you know, how do you coordinate teams with thousands of engineers distributed around the world where they're dependent, they're like code owner dependencies across different teams and you want to keep everybody unblocked and always able to kind of build on top of it, on top of their last change without having to wait for review. This model also, I think, is really applicable to working with agents because On kind of a shorter timescale, when you're working with agents that can generate code so quickly, the blocker really becomes like, how quickly can either you as the person directly prompting the agent or somebody else on your team, like, review all of that output and make sure that it's correct? And especially when you start to think about working with many agents, you know, working on a feature at once, how do you make sure that they are all, you know, unblocked to keep working together? Without having to wait for, for human approval necessarily. And that you have tooling that can go back, you know, say that they actually did something wrong and you have to adjust it. You have tooling that helps you apply those changes to everything else you built on top of it seamlessly without having to do a million rebase conflict resolutions. It's interesting that we're seeing, you know, we're seeing this concept that, you know, although it was initially developed to facilitate human-to-human collaboration, it actually now is starting to be, starting to become like perhaps more important than ever in facilitating those like human and agent interactions.

That's really interesting. It was a technology invented for organizations at the scale of thousands of collaborators, and that doesn't apply to a startup of 10 people necessarily until the advent of cloud code, at which point you do have potentially thousands of agents who are contributing simultaneously.

Exactly. Now you have thousands of tabs of your terminal open running cloud code. I mean, we talk about the window of companies for which something like Graphite is applicable has just shifted massively, where previously things like Stack PRs were only really valuable to 100 thousand-person-plus end orgs there, and really like multi-thousand-person end orgs is where it shined. But now it's like every team is becoming like that and every team is in turn facing those thousand-person challenges.

So let me ask you, we have the state of code generation today, which is you can expect Claude code to go off and do a task that takes, you know, let's say 30 minutes max, something along those lines. And it seems like we share a conviction that the review is the bottleneck. So writing code used to be 80% of the job. Now it's maybe 10%, 20%, not even that if you have an agent who can go off and do this for you. Where do you see this going in the future? And do you think that review will continue to be the bottleneck? Or I believe there's a term, yeet code, which you may have coined. The idea of you just have a merge queue and you say, go ahead and merge, and actually the AI is the one who's reviewing. Do you think this will continue to be the bottleneck in the future or will it be eliminated?

I think we will see the job of engineers, and we already heard from a few Graphite users and other folks talking about how the role of engineering is really shifting from writing code to reviewing code. As a trend, I think that largely continues where you're shifting from doing the IC work to being more of like the engineering manager or the orchestrator of that, of an army of, you know, more junior engineers that are writing code for you and delivering that to you. And I think the next phase that we're entering now is you're still like inspecting the code, you're still editing it and collaborating with those agents, but you're not the one who's, who's like originating or structuring that, like all of those, those pieces of code individually. So I think that, that, that change I think is here to stay. What I think needs to happen now though is just to meet the scale of changes that, that agents are able to produce, we need to just better focus our efforts as reviewers because otherwise what we hear from some teams is that, you know, they've given all their engineers things like Claude Code or CodeGen or Cursor, and, you know, now they just have this mountain of PRs that they're getting buried under. And, um, half the time, you know, the engineers who created the prompt, like, might not have thoroughly read the code. Like, it just requires so much more attention and cognitive load on the review side. And What a lot of what we're doing with, you know, with Diamond, our review agent, is we want to take off some of that cognitive load. And, you know, much in the same way that, that I think AI has, has let us work at a different level of abstraction on creating the code. I think you can do the exact same thing for review where, you know, you shouldn't have to focus as much of your time on reading for small logic errors or all the like nits and style inconsistencies that are so common in code review. The review should really be focused on more of the high-level concepts of, is this functionality implemented correctly? Does this experience feel good to use? Like, is this architected well? More of those like high-level senior engineer or product questions more so than like the nitty-gritty of individual lines of code. And using AI to focus on some of those more minor things helps us like direct our energy and spend less time ideally on every individual change and be able to just think at a, and operate at a higher level. For sure. Yeah.

So pushing this one level further, just because we're here to talk about, you know, AI coding. Basically what you've described is a scenario in which AI is better at handling the nitty-gritty details, the low-level things, maybe changes that can be fit within a single screenshot. Human developers who have more worldly wisdom, we can look at a codebase and say, "Hey, this is actually not the right architectural decision." Do you think that that changes in the future as we get ever-increasing models? Do you think that it's potentially the case that injecting humans into the PR review process is actually a liability to a certain extent? Similar to if you asked me to review assembly code, I would probably not know what to do with it. It's a good question.

I think that there's certainly, and we've already seen this from many anecdotes from customers where there's a class of bugs for which AI is actually better at finding them than humans. Like things that require you to like, you know, remember how a function is defined or in a part of the codebase you don't normally touch or remember a pattern that's not like top of mind for you. Like things like that, you know, where the AI can actually have more context than, than a human developer. It's actually a lot better at finding, at finding those issues, but And I think that we'll see kind of the window of, and just the scope of issues that AI is good at, is like good at detecting. I think that'll just ever increase human reviewers' attention on more of the functionality. You know, certainly I think it, and actually some teams have kind of adopted that like code model of diamond. We don't recommend this. This isn't kind of our official view on this, but some teams have gotten to the point where they've said diamond is good enough. Like if there are no diamond comments or if you resolve all of them. It's good to ship. And we've done as much as we can to make sure that every comment is actionable. One of the key differentiators between Diamond and many of the other AI code reviewers on the market is we really focus on actionability and making sure that if we say something, if we make a recommendation, it's something that actually leads to a code change. I think that's what, what leads some teams to be able to get to the confidence level of if you just resolve all of Diamond's comments, it's good to ship. But I don't think we're quite there. Maybe in a few years we'll be closer to that, that world where humans aren't really in the loop. But especially today and especially for some of the larger financial services companies we work with, for instance, as long as humans are accountable for the end business outcome, it's going to be very hard in the immediate future at least to completely remove a human from the review process.

That makes a ton of sense. I think an interesting engineering principle in general is that when some component of a system 10Xs, then everything breaks and you have to go through and address that. I think we're probably at that point with the actual generation of code. It's easy to generate 10x as much code. Therefore, the PR review process for actually getting that into production breaks. And so I'm curious to hear from you what specific aspects of the UX of the PR review needs to change. And I think we've been in this model for a long time where you have, you know, the GitHub pull request review. You go and you leave comments, you say this needs to be modified, and that assumes that, you know, a certain set of properties of the people doing the reviewing. So they have finite time, you know, they want to address comments one by one. Are there specific things that you think change about that with the advent of running 1,500 tokens per second on Cerebris with QEncoder in 2025?

A couple of things need to change there. One is just high-level understanding of what the change is. I think it's no longer safe to assume that the reviewer looking at this will have full context on the codebase or even that the author of that change will have full context on it. Having the tool, you know, be able to walk you through the change, to point out like what are the more interesting or like higher risk parts of a change and kind of guide your review. Like the fact that it's 2025 and the files in GitHub, you can't order them in a way that makes sense or tells more of a story. I think that that's something that's, that's kind of crazy to, to me and something that absolutely I think needs to change when, you know, when we're talking about getting to the throughput that agents can now generate of code. I also think the experience and one of the things that we're working on and have some pretty exciting announcements coming up around is like, how do we make it just more interactive? Currently the pull request is just this kind of static entity that you interact with and you go through your teammates or you add comments to, you resolve them. But how do you make it something that's more of an active participant where you don't have to go through that whole loop of get a comment, pull everything down locally, make the change, push it back up, re-request reviewers, like that whole back and forth loop is just so inefficient. And in a world where you can generate these, these changes in, you know, seconds or minutes, it doesn't make sense to have to go through that, that entire process, you know, just to make a minor change on the PR. So making it a lot more interactive, making you able to like ask questions of it, take actions on it, and have that all happen like in the pull request itself rather than having to hop between local and remote. That I think is a big shift that needs to happen, that needs to happen going forward. Last thing I'll say then is just as you just have a higher volume of these changes, like you need more coordination and you need an inbox where you can see like all the changes that are waiting for your review. You need automations to make sure the right reviewers are looking at the right pieces. You need a merge queue to handle merging all these changes concurrently and avoiding semantic conflicts. A scale of company or companies that are earlier in their stage and in size are just dealing with challenges of scale that are like several orders of magnitude higher than what they previously needed to, you know, just because you have so much more capacity to generate code now.

That's very compelling. One of the things about stacking that I find interesting is you mentioned earlier, it allows you to tell a story. And people typically think about a pull request, just a big diff patch, right? But actually there's basically a chain of thought that went into constructing that. And it's easy for a developer to spend maybe a couple days building a pull request. One of the things that Claude 4 and increasingly other models can totally nail though, is you take a diff patch and you actually break it up into the story and you auto-stack it of sorts. Is that something you guys have looked into or you're excited about for the future?

Absolutely. I mean, that's largely why we launched the Graphite MCP the other day. It's now available in beta and in the coming weeks it'll go GA. The whole idea is like, we think agents are amazing now at breaking up work into, you know, they plan work in checklists now. Like they work better actually when you have them first plan their work or break it up into, into like parts that tell a story and then execute it in that order. They can also go backwards and say, like, take this big piece and break it up into, into something that makes sense. And to then, I think the gap that we we wanted to bridge was like, how do you go from, if you already have agents that are working in this way and work and actually work better when they think, when you instruct them to go through that reasoning, how do you just then convert that into stack pull requests, which expose that thinking to the reviewer and make it easier for the right reviewers to look at it and tell the whole story of the change that was being made. And that's, you know, that I think is where we see, where we see stacking and agent development, like coming together so powerfully. MCP is very simple. It's, you know, part of it is just giving it the tool calls to use the Graphite CLI, but a lot of it is in giving additional prompting and instructions for how to think about breaking up the change into a stack and planning out how to do that most efficiently. So those are basically the two components of why, you know, what goes in there. You basically just turn it on and it just starts working. It's not, you know, there's really not that much to it.

I'd love a button in the Graphite UI where I can go in and click it and it's just, you know, stack my PR and it goes and it slices and dices it and sort of comes up with a narrative behind it.

AutoStack.

Is that a thing?

It's, uh, in the works. In the works. Uh, it's, we, it's very, very close.

Very excited about that one. So let's, uh, let's double-click on the MCP. I think, um, one thing I wanted to ask you about is this idea of CLI-CP. So it's been clear from the very get-go that agents are incredible at using CLIs and the Graphite GT CLI is one of the leading dev tools, at least in the GitHub adjacent, Git adjacent, utils that you can use. I can confirm independently, uh, you know, Claude 4 absolutely rips GT commands. If you just give it your token, you let it kind of log in. It doesn't really need much more than access to a shell in order to interact at least with the GT command line. So walk me through your reasoning here. Why is an MCP necessary? Why can't you just bake in that functionality into the GT command? Because it's already text in, text out. And what are the advantages of, of yeah, doing an MCP?

In many cases, just having, just having a CLI with good documentation is going to be enough for, you know, for vast majority of things that you're instructing something like Claude to do. There are a few cases where MCP makes sense though. I think one is where you just have a lot of function call of like different commands that you can use because, you know, we know that there's, if you give, if you give an agent a ton of different options, it can just be overloaded of like, okay, I have, you know, hundreds of commands potentially, like which ones of these do I actually use and what makes sense to use in what order? If it's particularly complicated and you want to have like a certain golden path or a certain, you know, certain set of workflows that you want to make sure that the agent is always doing. That's kind of one reason why to add additional prompting instructions in the MCP. Similar, but a slightly different variant of that is if, if there is like additional context in terms of how you should think about using the tool. And for us, that's something where we, we want to be able to give the agent more instructions on like how to think about breaking up a change into a stack and also to be able to kind of iterate on that. And like, as we push new versions of the MCP, especially like as there, as there are new frameworks or as there are new, you know, new ideas for like how software should be composed, like like, can we just make sure that we're always, we're always kind of taking our latest thinking on how to structure stack PRs and feeding that context to the agents? That's largely why, you know, why we decided to go the MCP route beyond just giving Claude the CLI.

That's interesting. One of the major advantages of command line tools is incredibly easy for agents to build them because the process of QAing it is just you run an executable on the bash shell and, you know, you're able to see the output. But an interesting thing that came up with Kimi K2 paper, I don't know if you had a chance to check that out, is a huge part of their post-training was scraping a bunch of MCPs from GitHub and then running them and sort of simulating what the outputs would look like. And so to the extent that you want to be able to take up headspace in future agents' training regimens, it does seem like it's a good idea to announce yourself as an MCP and people will hopefully scrape you.

Interesting. Well, hopefully we'll get some of the ideas of stacking into the training data for many agents to come.

Amazing.

Interesting.

So I think there's two more questions I want to ask you. One of them is around tracking AI agents' contributions. So previously in the sort of dark ages of programming before we saw the light of code agents, tracking an individual's contributions could be roughly approximated by the number of lines of code they have written. Obviously there's going to be some caveats to that.

Bad metric.

Bad metric. But besides asking their manager, it's kind of hard to come up with a better Maybe number of pull requests, but it's all these things are sort of coarse-grained, right? And especially if you have a model that is running inside your editor, it's kind of difficult to disentangle what the human's contributions were versus the AI's. Obviously, if you're stacking pull requests, there's like commit-level sharper resolution on who's doing what. So I want to ask, what do you think about how the tracking of contributions needs to change over time? What tools we have at our disposal to do that? Is this even important? What are the reasons that we should be looking at AI contributions versus human ones? And how do you think about that?

I've talked to some friends about this. Have a friend who prototyped essentially like get blame for agents or get blame for AI where you could see like what model was used, what was the prompt, was this from like a tab completion or from, you know, from like a composing or from an agent itself and having that included in get metadata. They're interesting ideas like that, but I certainly think that we, you know, we need to move to a world where we're storing, you know, not just what, what change was made, but you know, what model made it, like what was the prompt that was associated with it? Like as you move to a world where more and more of the codebase is regenerated, the idea of like the code diff itself becomes less important over time. And, uh, you know, what's more important is storing that output and then, you know, the artifact that was generated. You know, I think we're walking our way towards that, but certainly I think over time that, that needs to shift. The other thing I think is, you know, we are seeing now one of the big questions is like how much of this code is written by AI or like can be attributed to it. It's much easier. I think it's hard in the case where you have, you have like an IDE for where most of development is happening. It's easier when you have like fully autonomous agents where, you know, how much of that code is being written by them. And it's much easier to, to like attribute it to that. I'll be curious to see like if there are any best practices that emerge on, on that, especially everything is in flux so much right now. And I think the hardest problem is just that there are so many different sources of code. Each one requiring its own approach for measuring productivity.

Yeah, I know a lot of enterprise companies especially, they want to track productivity of agents so that they can justify the spend or so that they can understand which parts of the codebase should we be tracking more granularly and testing more because there's more AI contributions to it. I think you need additional Git artifacts in order to do that because it's not sufficient to just look at what the characters are that are in a given commit. I like the way you put it where You need to store the information that was used in order to produce the change as well as the original change. You can think of Git as maybe an append-only data structure. What you're suggesting is maybe you store the model, the hash of the weights that was used in order to— it's sort of the program that was used in order to generate the actual change.

Exactly.

That's very cool.

In a world where the output is less deterministic, I think what you need to move towards is more of storing the inputs that created it.

There's an interesting shift I'm seeing happening in analytics more generally where if you think of a bar chart or a pie chart, basically that is a ingredient that you would use in order to assemble a narrative about some action that you should end up taking. And I think the way that we've traditionally looked at tracking code contributions, we think about, okay, show me a bar chart of who our top contributors are, histogram over the last quarter. And one of the incredible capabilities, which I'm sure you've seen before of frontier agents like Claude Code, is Git archaeology, as I like to call it, where you basically tell it, hey, tell me what Meryl Lutsky's contributions have been in the last quarter and compare that to Jay Hack's or something along those lines. And it will literally go through and go deep in the Git log, figure out what's going on and come back to you with a natural language story as opposed to just bar charts. Maybe it'll justify that by showing you some additional information. And so I think when you're armed with this higher fidelity information on who's been making the changes, you're going to get this superpowered analytics that basically actually makes an actionable case for you beyond just a bar chart saying this is how you're using AI, this is what it's good at, this is what it hasn't been doing for you, all this type of thing.

It's also fantastic for teams in terms of identifying where the bottlenecks in their process because, you know, similarly, but you can go deep in Git artifacts. You can also go deep in all these logs from different systems of, you know, your merge queue, your GitHub analytics, your merge queue, your CI. And if you can feed all of that into Kontext, you're able to identify like, where are the bottlenecks in my development lifecycle and where should I, where do I have the biggest opportunities to speed up my team? And that's something that we've seen, we've seen some of our customers doing and having a lot of success with.

Well, so on a closing note here, I think we've gone through a couple different eras of software engineering. We had sort of like the analog era, we'll call it, where it was just humans and their biological hardware writing code. And then you had sort of the copilot era where we do autocompletion. The composer mode era is what I call it, where basically you have cursor and there's a little chat bar on the side or AMP code or something like that. And now we're sort of in the agents era. I guess you could call it background agents. You can give an asynchronous task, they'll go off and do it for you. Maybe what's next is the agent is slacking you and it's saying, hey, I've got this idea for a great feature. Can you please go get me a Graphite API? API key so I can implement it. I'm curious about where do you see this going? Are junior engineers going to have jobs in N years? Are senior engineers going to have jobs in N years? Is all software going to be written by machines? What does your timeline look like and how does Graphite's business strategy play into that?

Yeah, I think as you said, we're now in this era where we're moving to this world of background agents. I think that's the next 6 to 12 months that we're going to see large-scale adoption of agents that are writing code on remote compute, we'll see this big shift happening from— I think one of our core theses is like most of the development shifts from— on that timeframe, you know, next, I'd say maybe 12 to 18 months, like a lot of development will shift from local environments and IDEs to agents working, like generating code on remote compute. In that world, I think a lot of what matters and a lot of what is, what's so core is the review interface itself. That becomes the place where humans and agents interact. Agent is handing you back a code change, uh, it's handing you back a PR and you are then interacting with it, you're iterating with it. And that process where it's more like being an EM or being a PM on that project and, uh, like helping to like coach and refine, that's a lot of what software engineering is going to look like. I think then, you know, as you say, that next phase, the next like few years after that is going to be more of this, like I'd say agents or AI as like a remote agency or like a contractor where you give them like a very high level prompt and You're not even necessarily inspecting the code itself, but you're more looking at like, what is the end result? What is the end artifact that they've created? And is that giving feedback on that piece and they're, they're revising it. And then the challenge becomes less one of, of like, be like inspecting the code and being like in the details and more one of like controlling timeline, budget, you know, many of the considerations that you have in working with, you know, say an external dev agency or design agency or something like that. Last thing to your point around like junior engineers and senior engineers, I think there, there's both, there are two effects. Happening at once. Both of them are strong in opposite directions. Like, I think on the one hand, you have a lot of the, like, junior-level jobs that are being supplanted by AI. At the same time, you then have, like, this massive TAM expansion where so many more businesses are being able to, like, write their own software, create their own tooling. If you're able to, you know, use these tools, there are many, many more companies that are going to be looking for, you know, someone who is able to, like, effectively use dev agents. I think then you do really have to go either, like, like super deep on a technical topic or, you know, more broad and have like really strong opinions on like product and design, branding, like things like that, that give you some differentiated edge because like, if it's so easy to build anything, then, you know, the differentiator isn't being able to build the thing, you know, it's what to build and what does that feel like? And there are a million copies of what you're doing. Why does your product in particular stand out? And how do you get that in front of the right user base and actually get usage for that? That is a skill set is going to be one that's more important than ever in this next era.

Interesting. So you're saying essentially the job of the CEO, which is set the vision, the direction, is something that humans are going to increasingly move into while these sort of foot soldier-level details of fixing TypeScript errors and actually building features will be handled by agents. I use Deep Research all the time. I ask it, hey, go find me a market arbitrage and put together a business plan. And inevitably it comes back with something that's not very good currently. But do you think that that might be able to approximate human-level understanding of where the business opportunities are? And do you think it might be able to replace some level of human taste as you put it?

I certainly think we could, we will increasingly walk that direction where my friend, uh, Nikolay Rosan wrote this, this cool article like last year around like this idea of living software where it's kind of taking in its own logs and analytics and updating itself, like addressing the common pain points and updating itself. And you just kind of have to define what the right reward function or the metrics that you want to optimize for, and it'll just be able to update itself. Automatically. And I think you will see increasingly complicated systems that have that type of feedback loop. But I think that at some level, you will still need human taste and human, I think like human understanding and empathy of the problem in order to, to like define what those things are. It will take a long, long time, I think, before we're able to like fully replace, you know, have like an entirely autonomous company that's, um, you know, that's able to, to optimize itself and not require like any sort of human direction. Although there's the meme of like the one-person unicorn company, which We're still a long ways off from that, but I think that is directionally or asymptotically what you start to approach.

That's super interesting. What's on your wishlist for the back half of 2025 and 2026 in terms of agent capabilities, AI tools? Is there anything that's bottlenecking Graphite currently?

Obviously, we're always looking at what are the latest models, what are the reasoning capabilities, what is the window of context, especially for some of our larger enterprise customers. Like they have massive codebases and being able to ingest everything into context is still, that is still like a huge area of possible improvement for us. If you can get to a large amount of context without seeing the tradeoffs in reasoning capabilities that we see now, that's something that I think would be highly beneficial for us. Our biggest thing is like we just want to see more proliferation of background agents because the place of code generation shifts from the IDE to the background, the more important the pull request experience and code review becomes. The faster that our view of what the world looks like in the next year or so comes to fruition.

That's really interesting. Just double-clicking on that real quick. So I'm not sure exactly how you guys have it rigged up in the backend, but there's various ways which you can go about doing AI-generated pull request reviews. One of them is sort of a one-shot where you RAG stuff in, you try and pull in as much context as possible that's relevant, then you pass it to an LLM. It generates a review and maybe it points to specific lines it wants to leave a comment. The alternative, which is probably state-of-the-art if we're looking at just performance, and you'd know more about this than I would, is a fully agentic version of it where you just give the pull request and a codebase sitting on a hard disk somewhere and you ask the agent to go through and figure it out. Do you have thoughts on where the state of the art is going to be in the future? Obviously there's latency requirements, there's cost requirements if you're running this at scale that you have to take into account, but do you think that winning approach is going to change over time?

We went through that transformation at Graphlight where we started off just with RAG and we saw huge improvements to performance when we moved to agented code browsing. Giving the agent the pull requests and the codebase to browse and giving it a terminal and the ability to go through and, you know, grep through all the files. What I think the state of the art looks like is more, and where we'll get to, is more of like a combination of giving the agent the tools to browse, but then also giving it like superpowered search and almost like, you know, like a human using like a Sourcegraph or something. Like, can you give the agent the tools to more quickly find the right context? Text within the codebase, retrieve that quickly, and then take action based on that. That'll probably get to the point where it's no longer the bottleneck in terms of how fast we can review code.

Do you see that latency is a bottleneck to your users getting value out of the product, or are they okay waiting for maybe a minute or so as this thing agentically searches through large amounts of code?

Previous state of the world was you'd wait for hours potentially. So, you know, for us, if it takes an extra minute, it's still like a massive improvement over waiting for a human coworker. Versus like a real-time chat interface, you have this expectation of like every millisecond counts, like you need to squeeze all of that response time out of it. I think that that is something that works to our advantage. As we move more towards, you know, what I was hinting at earlier around like a more interactive pull request, then we start to run into more of the constraints of latency. And that's something that we're thinking a lot about from an infrastructure standpoint and from how we're designing our search and retrieval and generation of CodeComm.

That's very cool. And just last question for you. You mentioned Sourcegraph. I'm a big-time Sourcegraph user, a long-time fan. It seems as though the state of the art currently does not involve Sourcegraph. So you can basically hand ripgrep or, you know, find or some of these other bash commands to an agent, you know, like Cloud Code or Gemini CLI or whatever. And it's able to essentially replicate all the functionality you would get from a language server just by running bash commands. Now, obviously there's some things like running TSC, the TypeScript language server, or, you know, type checker. That it needs to do, and that can take a long time to parse a large codebase. Do you have thoughts on how tooling is going to evolve specifically with an eye towards being used by agents? Do you think we're going to see a new generation of tools come out that are more tailored towards them and people are going to RL on that? Or do you think that it's just going to be a full-on bitter lesson and we're going to basically end up with things that just look like what we had in the '70s, however made available to agents?

Always heeding the bitter lesson is, I think, is a wise decision. But I do think that there is this idea of like agent experience design that we're seeing now where, you know, tools need to be built with an eye towards like, how do agents understand how to use them? How do they more quickly access them? CLIs are great because you kind of get like great UX out of the box versus on a lot of like web experiences, APIs that don't necessarily have that same thought towards how an agent is using them. You know, for code browsing, there probably won't be that much of a, of a delta there, but certainly I think for for many other tools, a big shift in how we optimize for it. It is an interesting thing to think about what happens then when the agents themselves are trained on how to use the tools and some interesting effects that happen in that interaction. But I'm curious to see how that all plays out.

Yeah, that's right. I mean, in a way, the better lesson is just do whatever Anthropic has RL'd their models on right now. And they did not include source code, as far as I can tell. And so it's not super skillful at that. But we have seen examples. So for example, OpenAI's recent victory in the IMO, up the gold. And it seems like this was a result of just aggressively RLing. Amazing achievement. Aggressive RL. DeepMind also deserves credit for achieving it and maybe using similar methods. And you can imagine something where it's like, what is the IMO gold medal of coding? Well, you give it sourcegraph on crack, it has access to everything it needs, and you just let it run with those tools ad infinitum and it will probably be able to build some pretty significant things. The bitter lesson is you just got to train bigger models and give them good tools.

I think so.

Well, thank you so much for taking the time and hopping on here with me. Really interesting to hear about everything that's cooking over at Graphite. We're rooting for you here at CodeGen. We, you know, see diamond reviews pop up on our pull requests every now and then. The agents will sometimes chat back and forth. Yeah, that's great. Yeah, they're complimenting each other. Like, I love your use of commenting there. It's really great. I love that you're absolutely right. Yeah, exactly.

Yeah.

Something got added in, in Claude 4 where it loves at the end of completing a task, it'll spend about 2 paragraphs just complimenting itself and saying how good of a job it did. I'm not sure why that's the case. Maybe that's just Anthropic trying to juice a little bit more tokens out of each request.

How much money is spent?

We had the "how much compute is spent on thank you." We need the "how much compute is spent on that's absolutely right." I love the idea of some PM at Anthropic thinking about how they can juice a little bit more by getting review bots and code agents to interact more aggressively. They just somehow, you know, encourage that behavior.

Yeah. The wrong— somebody's optimizing for the wrong metric and we're just gonna end up with like, you know, 100 comment threads on PRs now.

But extremely polite, uh, you know, comment threads.

Yes.

That's amazing. Awesome. Well, thank you so much for taking the time. It's been a pleasure speaking with you.

Thanks for having me.

And, uh, yeah, looking forward to see what you guys roll out soon here.

Appreciate the conversation.

Thank you so much.