Coding startups can't beat Anthropic at their own game – Harrison Chase, Co-Founder/CEO of LangChain

When I think back to the capabilities of early 2023, you really couldn't get something to work that was more than like a couple thousand tokens. It just would kind of fall off a cliff. And then maybe around GPT-4 era and then Claude Sonnet 3.5, capabilities really got stepped up.

I think we'll start to see longer and longer running, more complex agents that can act over longer time horizons. And so one of the things that I've been really excited about recently is what I call kind of like deep agents. The whole reason that people would use a framework like LangGraph in particular that gives you out of control is that you want to guide the LLMs, the agents, in a particular way.

That's how you go from $10 to $100 mil in 7 months is you have the product pre-built, model shows up, we're good to go, y'all. Hey Harrison, so glad to have you on the podcast. Thank you for joining us. I think I might actually have a claim to be one of the earliest users of LangChain from 2, 3 years ago, something along those lines. So it's a pleasure to have you here and ask some deep questions about what you guys have been up to. So maybe to start us off, can you give a brief introduction and talk about the many phases of LangChain that have brought you to where you are today.

Yeah, absolutely. And thanks for having me. And yes, you were definitely one of the first people on Twitter that I interacted with, and that was, that was a lot of fun. LangChain started off basically on Twitter. It started as a Python package that I worked on nights and weekends and then basically tweeted about. This was a month before ChatGPT, late January 2023. That was probably the first phase, and that was just building the Python package, tweeting about it basically every day, doing a bunch of integrations, doing a bunch of use cases, all of that stuff. I do kind of think companies kind of go in stages, and so the next stage was kind of like February 2023 to like summer 2023. And we built our first commercial product, LangSmith. We grew the team, we raised 2 rounds of funding. In August 2023, we started selling LangSmith for the first time. You know, the next phase of the company was building LangSmith, the product, LangChain keeping on going. And then in early 2024, next phase of the company, we launched LangGraph. That's another open source project that we do. Agent orchestration, much lower level, builds on a lot of learnings from LangChain. we bring on 2 salespeople to help sell Langsmith because we're seeing a lot of traction there. And then, you know, maybe fast forward another year and, and beginning of this year, 2025, we start building out our enterprise sales team a bunch, bring on a VP sales, like Langsmith testing observability. I think most people know it's something that you need. Most people don't want to build it in-house. And so we're seeing a lot of traction there. But then also LangGraph is going well. And so we start building kind of like LangGraph platform. I mean, that's a deployment platform for some of these agents. I talk about all the products, like I spend a lot of time on product these days. That's what I think about. That's how I think about the company and the journey. I think a lot of our product story really does actually map the journey of people building in the space. First, you kind of start with prototyping, and that's probably what LangChain was and still is, is like the easiest place to get started. Then once you've got a prototype, you need to iterate on it until it's good enough. And great, like that's where Langsmith comes in with a lot of the debugging and testing. And then at some point you want to do some context engineering and you want to control the architecture. And great, that's where LangGraph comes in and it's got all that controllability. Then once it's good enough, you go to deploy. Great, that's LangGraph platform. We've got something there. And then after you deploy a bunch of them, there's this management aspect to it. And that's where we're building, or that's where we will be building a lot of new stuff. And so I think, yeah, our product journey kind of like nicely maps to what I see kind of like happening in the space. And, you know, now compared to, again, just like thinking of that journey, early 2023, people were just prototyping. Now we've got, you know, hundreds, thousands of agents in production. So I think it kind of nicely maps there.

Yeah. You know, when I think back to the capabilities of early 2023, really, couldn't get something to work that was more than maybe a couple thousand tokens. It just would fall off a cliff. And then maybe around GPT-4 era and then Claude Sonnet 3.5 capabilities really got stepped up. I know that the sort of LangChain to LangGraph transition was driven by the fact that you suddenly could run an agentic loop and it could do tool usage for a certain number of iterations. I'm curious, where do you see this going next? Are there going to be new capabilities that are unlocked by the next generation of models that has some implication on what we should be doing in our agentic platforms? Is there going to be another shift like that?

I think we'll start to see longer and longer running, more complex agents that can act over longer time horizons. And so one of the things that I've been really excited about recently is what I call kind of like deep agents. And so I think you see this with like Claude Code. You show someone Claude Code a year ago, they'd freak out. That's amazing. Same with Deep Research. You show Deep Research, you show Manus, another great example. All of these are projects that can do things over longer time horizons. And deep agents is kind of like, it's an open source package/blog that I wrote to kind of like encapsulate what some of the common patterns that these things are doing. And right now, basically, they They do planning, they spawn subagents, and they basically create files as an artifact of them kind of like working. I do think we're moving towards more and more of these long-running complex agents actually working and actually providing value. Will the same tenets of that planning, of that subagents, of that file system still be useful when the next generation or the next, next generation of model comes out? That I have less— like that I don't know. Like you could argue that a lot of the planning will just inherently go into the model. That's not at all been the case to date. Like Claude Code, they build the model. They also use Claude Code with a planning tool. It's clearly kind of like providing value today. Will some of that start to disappear? I feel like the file system stuff will always be around. Even as context windows get longer and longer, I still think you're going to want to manage some of the context externally from kind of like that short-term context window. Could be wrong, but that's kind of like, that's one of my stronger beliefs. Subagents, I think, are generally useful again for running things in parallel. Maybe planning, they just get better at planning.

Yeah, it's interesting to think that the file system is sort of like the catch-all data structure for writing memory to. So you can write a todo.txt. I see Claude Code does that all the time. You can construct a program that will store stuff in a SQLite database for you if that's what you want. And so it kind of relieves the agent implementer from needing to come up with the proper data structure and you can kind of just offload it to the model.

Yeah, it's really interesting. It gives them a bunch of flexibility, but then all the difficulty becomes how do you prompt the model or the agent to do that? In a reasonable way. Like, you know, because the types of notes that I might want to take in a history class are pretty different than the types of notes that I might want to take, like, in a math class or something like that, or the types of programs I might want to write. I think that as humans, we build up this, like, tacit knowledge of how we want to, like, take notes. And, like, will that be in the models, or will we have to prompt that to be? You look at Claude Code's system prompt, it's 2,000 lines long or something like that. Like, you know, or the leaked system prompt or the reconstructed system prompt or whatever you want to call it.

Yeah, it's interesting that so much of this depends on basically what Anthropic decides to train it on. So if they trained it to write its memories in a certain way and it was using the file system, then congratulations, that is now what you should bake into LangChain. That makes the most sense. That's what's going to be the best performing thing, right?

Yeah, absolutely. I mean, Anthropic has this file system tool, I think, that they train the models on for editing files. And when we released something today called OpenSUI, which is like a coding kind of like agent, as part of it, it can edit files. And in intuitively, if I would've thought about how to edit files, I would've thought like, okay, give it a tool where it replaces line, like it can specify the line numbers and the new things. But that's not what Anthropic's models are trained on. What it's trained on is replacing an old string with a new string. And so that's the tool we gave it because that's what it's trained on and that's what it's gonna do best on. And so yeah, 100% trying to use the built-in tools that the model is trained on. And by built-in tools, to be very clear, like I don't mean actually, like they're not running, well, sometimes they are running tools behind the scenes, but this like file edit tool, it's not actually editing the file, it's just generating the payload to edit the file. So it still runs kind of like within your agent loop or whatever. But using those kind of like built-in tool definitions to generate the payloads, I think makes a ton of sense.

Yeah, it's interesting that the Claude code binary comes bundled with like a precompiled version of ripgrep for Windows, Linux, and Mac. And so it actually does have a tool that comes with it. So I mean—

I didn't know that.

Yeah, it's fascinating. I guess my Mac doesn't have ripgrep already installed, but it's simple enough. You can compile 3 or 4 of them. Do you have any hot takes then on if Anthropic decides to train something in this very specific harness that is Claude code, it's obviously going to be better at using that. What is the play then if you're a generalist agent framework implementer? Are you just going to take the L and say we're always going to be only 95% as good as Claude code, but we're going to be more configurable? Do you bake Claude code in directly into the actual framework? And so I can use a subgraph in LangGraph that's going to be the Claude code node. How does that look?

100% on the latter one. Yeah, 100% that I think the multi-agent stuff starts to become pretty interesting. I mean, I think the whole reason that people would use a framework like LangGraph in particular that gives you a ton of control is that you want to guide the LLMs, the agents in a particular way. You're doing context engineering, you're teaching it to kind of like do something. And so I do think for like things where the model is really trained on, like code, I mean, I think it's a really interesting question of what happens to companies in the coding space. I think like Anthropic has this model, this program that's really trained on code. Do you build around it? Do you build tooling for it? If Anthropic does go super heavily into code, trying to build a coding system on top of Anthropic that's different seems really hard to me. Luckily, most of our users are not building coding systems. So in terms of like how we think of enabling people, it's really like, honestly, if the models are really good at this, the model labs are probably going to come out with a solution in the space. But like there are going to be a lot of instances where you're going to want to build into your product something to do with your data or your business workflow or your mental model of the world. And so how do you teach/train/coach these agentic systems to do that particular thing that is different than what the models have been trained on? And I do think there's some overlap where, like, the file system tool, you can use the file system editing tool for more than just coding, right? And so I actually think one of the best places to think about using it is for research, actually, like take notes as you go along. Like, I actually think that would make deep research things a lot better. And that's different than coding. So there's these overlaps. If a lot of the model providers go really deep in a particular domain, which seems to have happened with code, it's going to be really hard to compete by just building a better version of just Claude code. So you need to differentiate either in terms of integrations or how it plugs into the workflow or infrastructure around it. But there are a ton of use cases that aren't code. And I think that's what most of our folks are building.

Yeah, there's some interesting takes there where, for example, AMP code, a number of things going on, but one of them is just we are going to use more expensive models than Claude code. We are going to rack up more costs. We're going to shove more in context. And it's a slightly different customer profile. We've certainly heard from executives at various companies we work with. They're like, I don't care if it costs $100 $100 per ticket that I end up resolving. I just want the right answer. I think that's the type of thing that Anthropic could hypothetically have a mode you would switch it on that's like ultra max mode, but that's probably not what they're going to spend their time on. What do you think about that?

I think a lot of the differentiation in my mind will probably come down to how you put this in the user's workflow. If you're building a terminal coding agent, which Claude Code is, and you're focused on that, I think that's going to be really tough. If you're spending, like, I think you guys are doing a good job of bringing it into Slack where people are and having it kind of like run. And correct me if any of this understand, you guys have a lot of great billboards in SF, so I'm basing it a lot off of that. But like that to me gets into like the UX of these things. And if you think about Cursor and why Cursor took off, I would argue it took off because of the UX and they really nailed the UX. They did a great job of thinking that through. And so I think there's a ton of like really interesting thinking to be done at the UX layer of how do you bring these longer running things into your workflows where you still can like interact with them, but they, you know, can maybe run off kind of like in the background or something. I'm not saying like there's nothing interesting to be built in the coding space. Claude Code is basically an agent. Cursor early on was not an agent. It was an IDE with a bunch of things kind of like built in. And so we try to build a lot of stuff in the open source on our applied AI team that we think is representative of what people build and what people want to build. Like we never built like an open source version of Cursor or anything like that because there were so many integrations and so many things that, that were difficult. When Claude Code and a lot of these ambient agents that came out, started doing well, that's basically just an agent and we build agents. And so we're like, oh, okay, this is really interesting. We should just build a version of this because a lot of the hard work that isn't agent related is around all this UX related stuff. And we want to focus on just building the best agents and the tools for that. Where I'm going with this is Claude Code is more like an agent than a product in some sense. They do a bunch of cool stuff in the terminal. This also depends what Anthropic wants it to be. Agents by themselves are not necessarily products. But if an agent is the product and Anthropic's building an agent that is the product and you're trying to compete directly with that without investing in all the other stuff that goes around the agent to make a product, that I think is really tough. So I think that other stuff is actually like the UX is where there's a lot of interesting things.

Yeah, it seems like if you're doing a pure agent implementation, it's a bit of a race to the bottom. You're going to have extremely low gross margins and you're going to be competing with the exact people who are selling you the underlying infrastructure.

But the reason we try to do it is I think there's a ton to learn there that you can then bring to other agent kind of like types. And so I think that's a lot of our motivation.

So you mentioned the file system earlier. I think actually just like having a Unix machine is incredibly valuable outside of just coding. And Monas is a good example of this where when you have Unix commands, it gives you composability of tools, which you don't necessarily have otherwise. So you can pipe, you know, one thing into another thing. You can, you know, use head and tail in order to slice and dice and like zoom in on windows of your output. I think that's something that I have yet to see somebody else realize in a fully generalizable way. Plus there's infinite data online, so these things absolutely rip bash commands. I would be surprised if Claude Code isn't sometime soon maybe one of the best editors of prose or one of the best agents for updating your Salesforce just because it can actually pipe a bunch of these different tools together.

One of the big questions in my mind, and I'd love your take on this, is when we have these agents running, are we going to give them their own virtual machines to run on, or are they going to be running in a centralized place and connecting to other things? And what I mean by that is, if you think about file system stuff, you could have each agent have its own virtual machine with its own file system and just run there. You could also have, quote unquote, a virtual file system, which is actually what we did for our implementation of deep agent stuff, where the files are just stored in Postgres and you just write some thin wrappers to mock out ls, edit, Do they each have their own machines or are they running in a central place? I don't know if you have a take on this.

Yeah, I do. I think that the deeper we get into this conversation, the more I'll just end up harping on the bitter lesson, which I think I've learned several times over the course of building this company. I think what the bitter lesson would have you say in this scenario is let the agent decide. So there is a scenario in which there should be some private information, but the agent should also be able to upload content to a shared file system or mount a shared volume. So I saw that OpenSUI is using Daytona. I believe they have some affordance there where the agent could say, take my private sandbox and put this instead of on my "/user/whatever" and put it in the shared space so other agents can see it. Or another version of having a shared file system is you push something to a Git remote and so other agents can see it. I absolutely think that an agent should be able to spawn a subagent inside of its own sandbox. So it's sort of like we have shared state, shared context, shared artifacts on our machine. But at the same time, I think there is a use case for having fully separated agents as well. And you could accomplish that with Git worktrees. But I think basically the most reasonable default, at least today in 2025, is you basically have each agent start on a separated sandbox and allow them to mount some type of an external file system. There's obvious infrastructural benefits to having everything on a single machine. There are benefits to not having everything on a single machine because it allows you to do snapshotting, especially if we're talking about coding agents. So if you have one agent you want to store its entire state, including the memory potentially if it's running a server, that's hard to do if you have 4 different agents on one machine. I would say that there's always this question of, okay, what's the long-term right thing to do? And then there's the question of how do you win in August of 2025? The winning hand today is basically to use one of the existing sandbox providers that provide snapshotting. And typically the best way to do that is to have separate agents on separate sandboxes.

And I wonder how much, like, do you think this goes beyond coding? Like if I'm building a legal kind of like agent, should I have basically a virtual environment for each agent where it runs and it's got all its stuff there? So ChatGPT has their like code interpreter thing. And my understanding is, I guess there's a difference between like the agent running in the sandbox and the agent running and connecting to a sandbox, right? If that distinction makes sense.

Sure.

Does that distinction make sense? And so like, do you—

Yeah, basically what we're, we're discussing is permissioning. So what is the right permissioning model for agents? And I think that we could ask the same thing about humans. You know, should you be able to access everything on my MacBook or should you be able to see my leads in Salesforce? And I think this is a huge problem. There'll probably be entire companies that are built basically to address how you can enable agents to request permissions from other agents. Maybe there's going to be one daddy agent who ends up assigning permissions to different agents and saying, "Yes, you can access this guy's work." I think it's very difficult to come up with an answer to that without thinking about the context in which the agents occur. If it's a healthcare context, then there's going to be pretty stringent requirements around that. You just can't trust even Claude Opus to not request to see somebody's medical data they shouldn't be able to see. I'm not sure that there's a one-size-fits-all thing besides saying that there should be configurability. Maybe that's where LangGraph comes in. You guys could build in something along the lines of a permissions model where there's an agent that can request permissions that ends up getting rebuked, something along those lines. Another interesting question around this is we've talked a lot about sandboxing and what's the optimal way to arrange that. What about compute? We have these new processes. It's kind of a different shape of computing than we used to have where you have a very long-running process. It's largely I/O bound, so the vast majority of the time of an agent running is not the actual Python for loop, but rather it's waiting on the LLM to get back to you or waiting on the sandbox that you're operating against to execute a tool call and return it to you. I've seen a couple of things pop up that are more tailored compute for agents. Vercel has an offering, Modal has an offering, a couple of other folks. Ingest, I think, is one that targets this. I know LangGraph or LangChain has its own offering. LangGraph Cloud, is that right?

Our naming is terrible. I think it's called LangGraph Platform in our docs, but we've called it LangGraph Cloud in the past as well.

Okay. If the CEO doesn't know the name of it, then you can't blame me. So do you have thoughts on how this sort of— is there going to be an emergence of a new computing primitive that specifically services agents? Are existing EC2 instances sufficient for this? Do we need new abstractions in order to facilitate agents?

Yeah. So I think the idea of an agent runtime's really interesting. At the end of the day, I think like a lot of agents are long-running processes that are stateful. In some way that could be like long-term memory, short-term memory, the conversation history, the file system. There's some state and they generally run for a period of time. And so what, what is like the right way to do compute in long-running stateful kind of like situations? And I think like this is generally, I would argue, more difficult than other situations. It's not an entirely new problem. So I think Temporal has, has been around in the software world for a while and they're solving this very hard problem for traditional software types of applications. And the reason Temporal exists is because this doing this and doing this reliably is hard and they're good for that. I do think there are some differences between— I'd put ingest in as kind of like a Temporal equivalent. So doing long-running stateful applications for agents, I would argue, has some differences from kind of like doing it from without agents, from kind of like the Temporals, from the ingests of the world. You can absolutely use Temporal and ingest for kind of building agents. I would argue that there are other primitives that would be good to have built into kind of to, to a runtime like this. So specifically a lot of what we do in LangGraph Platform, it has the same like durable execution that comes with Ingest and Temporal and you know, that they've made famous. But it also comes built in with really good streaming support, like streaming's really important for a lot of these agents. And you can do that in Ingest and Temporal and other workflow situations, but you have to write it by hand. You have to write it yourself. We come with it baked into the box. Human in the loop and human on the loop thing. So being able to basically stop at any point in time and then, you know, get some user feedback and resume or go back to kind of like a snapshot. I think you can do the human in the loop bit with the other things, with the previous solutions. I don't know if you can do the human on the loop thing, like the time travel-like aspect.

What's the distinction between these two?

So human in the loop would be like an agent, like, you know, any, an agent takes 10 steps, it like writes 10 different files and then it's like, hey human, permission to delete this file or something like that. And so there you basically wanna pause indefinitely until the human comes back and says yes, permission permission to delete this file. And that could be 1 second, that could be a day, you know, that could be a year or whatever. Human on the loop would be like allowing the human to come back and basically go back to like step 5 and then edit the state from there and then resume from there. So say like, hey agent, you actually messed up at this point in time. And so that's what we call kind of like human on the loop or time travel is like you've got the steps, the human's like a little bit above it, peering down, able to jump in and out. So LangGraph platform supports all of that. And then I think short-term memory and long-term memory in this like persistence layer, that is basically the scratchpad for agents kind of like working and humans kind of like sending messages. At the end of the day, these are just databases. I don't want to try to like overcomplicate anything, but like LangGraph platform comes with these built in. So I do think there are primitives for agents that, that determine a little bit what an agent runtime looks like. And I also think it's super early on here. So I think there's only going to be more and more kind of like differences as the space matures.

Yeah, for sure. So you could rig it up on top of, let's say, Temporal to do all of these things, but that's just a lot of extra steps. And you could also do pip install LangGraph. You're off to the races. LangGraph deploy, how do I deploy? Remind me.

We need a LangGraph deploy. Right now we have a one-click deploy. So you go into LangGraph platform, you point it to a GitHub repo. We need like the modal deploy, the LangGraph deploy. It's, yeah.

Yeah, honestly, there's no excuse for not having a good CLI in 2025 because that is arguably the best target for an agent to write because it can just execute the code right there, write unit tests for it, so.

That's true, yeah. Yeah, I mean, I've seen a few tweets about, you know, how we gotta think about like, agent development experience. What does it look like for an agent to use tools? And yeah, that is a new thing.

That'd be a great CLI. I think that actually may be a cold take at this point, but a lot of people have really pointed to MCP as saying this is, you know, our salvation for agents. You're not going to have to implement your own tools in the future. You're not going to have to inherit BaseTool and LangGraph. You can just use these external things. And to me, it's always just been the CLI. Like you can write a CLI for GitHub or a CLI for Sentry or something like that. The agent already knows how to use that. You just need to do pip install Sentry or pip install LangGraph and you have something that's good to go. Plus you get the composability of other Unix tools like head, tail, piping, this type of thing.

I think that works really well for coding because you're already in a coding environment where you have a terminal. I think an open question is again, like for legal, like do I want to give my legal agent access to a coding terminal because that's best for whatever reason? Or will like doing things via API just be the easiest way and more scalable way and cheaper way and whatever to do it? That's an open question. I actually don't know. I think for coding, pretty clearly, doing stuff in the terminal is where the space is at right now, at least.

Maybe one reason in favor of it being done through the CLI is because we're going to see such a rapid increase in the pace at which CLIs get developed because agents are good at developing them. You're going to see basically that take off faster than MCP. You can imagine any tool that you use, LexisNexis or something along those lines for legal, there is a CLI equivalent of it. Ultimately, it's text in, text out. But it comes with the composability. I could see it. I think Monus is actually a good example of this. They have a bunch of video editing examples, right? Where it comes in and I don't even know what it does. It does some crazy video editing. And it's literally just piping FFmpeg commands back and forth and doing a bunch of slicing and dicing. You could imagine coming up with an iMovie MCP tool. You could do that. But ultimately, iMovie is effectively a wrapper on top of FFmpeg and maybe some other tools like that.

Well, you know, I do want to write a CLI. I want to create an example of a coding agent using DeepAgents, and I want that coding agent to be a CLI tool. So of course I'm going to use it to create itself. And so I'll let you know how good it is at writing CLI tools. I'll find out firsthand in a few days or however long it takes me to get time to do this.

It's funny how people used to talk about, you know, can you get a language to compile itself? So write a compiler that's capable of compiling itself. And this was sort of like the test of like, oh, you know, how good is a language? How long is the minimum script that can compile And now it's sort of like, what is the minimum length of a coding agent written with your platform that can end up reproducing itself? Non-deterministic version of it. So last one I want to get your take on. You have been a thought leader on memory. I've seen a lot of stuff coming out of the LangChain shop regarding how to get agents to remember persistence over time. You talked about snapshotting a minute ago. This is something I've always been a little bit skeptical of. And the reason I'm skeptical of it is I've seen in production with agents like Claude Code, even if you give it no prior context on a certain issue on a very large code database of how to solve it, has no access to memory. It already is so good without access to memory because in some ways, the memory is just the Git tree. I would be curious if that's the same in a legal context. Maybe the memory is just the set of all Word documents that the organization has access to and agents are very resourceful at going out and reading the things that are relevant to the task they're doing. So what to you is the most compelling application that exists to date of memory? And then sort of follow-on question, where do you think it's going to go next or do you think it's going to increase in importance over time?

I don't really think there is a compelling application of memory today, which is why we, I don't think we've made much progress in memory. We spent a lot of time on it, even like a year and a half ago, we came out with like a service called LangMem, did a few POCs around it, basically found that until the agent was like functional, that was the priority. And so like to your point earlier, like until cloud code is good, the lowest hanging fruit is to just make cloud code better. At least a year and a half ago, that's where people are at, where the lowest hanging fruit was just improving the general capabilities of the agent. Personalization memory, are those the same thing or are those different? Great question. TBD there as well. There's maybe like two types of memory. One is ingesting kind of like a bunch of data ahead of doing work, putting it in some data structure and then retrieving it. And I actually think you do do this somewhat in coding as well. I know Cloud Code doesn't, I don't think, but Devin, I think famously does. So they have their kind of like deep wiki thing where they construct a mental model of like the repo, not a mental model, but some data structure representing the repo that they then use when they're making changes. And so I think that's an example of kind of like the this clone your brain type thing where you do this big offline job, create a data structure that's basically created by running an LLM over the raw data and getting this new thing. Then I think another form of memory is like, as you're interacting with the agent, can it learn things from that? And Cursor has talked a little bit about how they do that. And right now that's very basic. They basically just update like essentially a file that's put into the system prompt. It's similar to like, you could think about it as like updating ClaudeMD. And so you actually could argue, I mean, one, I think interesting take on memory is that it's nothing more than a file. System. Again, this is where it's so interesting. Yeah, you just, you just have an agent running on a box and it just manages its whole memory and all of that. I do think a lot of this stuff only starts to become important after you have some base level of performance. Like Cursor didn't start thinking about memory until they were already at like, you know, $300 million ARR or something like that. The other folks who we talked to about memory have something in production that's working. And so I do think it's a second order thing. I think it will get more important in the future as more things get functional. I don't know what that will really look like. A lot of the difficulty in memory right now is prompting the LLM to recognize what to put in memory so it doesn't put too much or too little. And so I don't really know how that will get solved or get easier. And so yeah, it's a really interesting but really confusing space as well.

Interesting you say that there's just so much low-hanging fruit on making Claude better. I think we've seen that from the foundation model side as well. So there was a period maybe 3 years ago where there was a lot of people who were publishing work where there was somehow an external data structure baked into the LLM. One of the that come to mind are Stack RNN, there was something called Retro that you could take in a very large corpus and that could effectively be memory. None of it worked. None of it ended up making it to prod. But you could totally imagine a version of the OpenAI APIs where you somehow pass in a scope key. So it'd be like organization equals LangGraph and then it would basically be able to store and retrieve memories maybe in a fully differentiable manner from just weights or something along those lines. So it wouldn't really be up to you to come up with the mechanism by which it stores memory. Do you have thoughts on that and if that could be coming down the pipes?

I personally think memory will probably live outside the model. It's a good chance it could be inside the model provider APIs. I mean, you look at what like OpenAI is doing with some of the tool usage stuff, and now even Anthropic, like you can call search tools through— as when you call the model, it will not only like generate the payload for the search tool, but it will actually execute the search tool. But that search tool still lives outside the model weights, right? They're running an agentic process behind the scenes to do that. I think that's what it will look like for memory. I don't think it will live within like the model weights itself.

Is that for auditability reasons or because you think it's easier to train that way?

I am definitely not an expert in training models, but— and it could be that I'm just optimizing for kind of like short-term what I think will work best and not, you know, adhering to the, to the bitter lesson where you just, you know, go for simplicity and try to bake it in. But from my understanding of the space, I just think it will yield better results if it's some data structure that lives outside.

Certainly easier to stomach. If you can see what your agent has remembered and there's some mechanism that you can delete things, so if you need to implement zero data retention, but the actual memory is baked in some non-deterministic way into the weights, it's very difficult to get out of there. There will almost certainly always be a place for external note-taking. I guess the question is, is there also some level of system 1 memory that this thing builds up where it's a reflex to do something properly?

I do also really believe in in-context learning. If you want to say something like prompt caching on really long context is memory, then okay, yeah, I could see that. But I would argue even that, that is snapshotting the forward pass. That's also kind of like a system around the model almost. But I'm actually a big fan of in-context learning. It's basically prompt caching, like prompt cache on this thing. And if there's a way to prompt cache on millions of tokens efficiently, I think that would be very interesting. That would probably be something that would be pretty tough to do. I mean, you can't really do that without the model weights, so that would probably actually be closer to the model. I guess what it comes down to is I really don't know, but I think there's a lot of interesting stuff here, so.

Well, hopefully whatever comes down the pipes, we see it baked into Landgraf. Hopefully. So last question for you, what are some products you really admire?

This is not really a product, but I think the skill that goes into kind of like cooking is, is really impressive. You know, in so much as the, the plate of food that someone produces is a product. Like I, I have a lot of respect for kind of like that line of product work. Maybe more in the tech world and what we use internally, Notion, Linear, Figma. I think these are all pretty standard things. I'm trying to think of a more interesting angle to one.

Those are good picks. That is sort of the Mount Rushmore of UX.

Nothing interesting, sadly. All pretty chalk. I picked all the one-seeds, right?

Usually people say Claude Code. This is obviously a bit of a cop-out if we've been talking about it this entire time, but yeah, no, Claude Code is very good.

Yeah, I mean, in terms of agentic things, Claude Code's great. Cursor, I think, nailed the UX side of things. I think, yeah, a lot of respect for there. Devin, I think, was way ahead of its time in terms of kind of like autonomous async kind of like agents running in the background. We took a lot of inspiration for how we think about UX from some of their kind of like product videos. I think Replit does a really good job with like human in the loop. They have all these sandboxes already. Like they talk about right time, right place. Like they'd built all that infrastructure and now they kind of like just layer an agent on top. Like that's fantastic.

That's how you go from $10 to $100 mil in 7 months is you have the product prebuilt and you just, oh, model shows up. We're good to go, y'all.

Exactly. Um, Allbirds. I love Allbirds shoes. Um, they're super comfy. So that's—

Do you live in San Francisco?

I do. Yeah. How could you tell?

I heard somebody refer to Allbirds as Salesforce Ones. It's like the Air Force One, but for, for tech bros like you and I, technology brothers, excuse me. Yeah.

Very, very spot on.

Awesome. Well, hey, thank you so much for taking the time to hop on here. Fascinating to learn about your perspectives on memory, compute, agents, Claude code, all things related to AI. And we're looking forward to see what LangGraph puts out next.

Thanks for having me, Jay.