How Factory builds agents that help across the entire SDLC with Matan Grinberg, Founder & CEO

One principle that's very clear is everything needs to be written or recorded. Because if we want our agents, if we want our droids to make decisions as good as our decisions, they should have access to the same information. So if we're having an engineering sync, that needs to be recorded and the droids need to have access to it. If we're meeting with a customer, it needs to be recorded. We need to have a document about, you know, our thought process and our discussion there. Slack conversations. Droids need to have access to all of this. There can be nothing that's like tribal knowledge or that only exists in people's heads. Because that now means you have a discrepancy between what your agents can access and what your humans are aware of.

Matan, I'm so excited to have you on here.

Thank you so much for having me. It's a pleasure to be here.

Of course. I think we first met at a Factory happy hour. I don't even remember when, a few moons ago.

Sounds about right.

And we've seen each other many times since, but we've not had the chance to get into the meat of Factory, so I'm looking forward to doing it today.

Yes, it's a pleasure.

You know, cogen is a very popular topic. That's probably an understatement. In this LLM wave, developer tooling has seen incredible success in generating revenue. I would say early on, you know, before a lot of general population was talking about LLMs, GitHub Copilot was printing real money. And so Factory was founded when foundation model companies were working on better cogen. Startups and large companies were working on dev tooling. I'm curious how you landed on your vision. There were a few things that you did that were, that were different, like an AI engineer in every tab, agent-driven development versus building an IDE, enterprise-focused versus the vibe coder. What gave you conviction to build a company in this relatively crowded space and in those directions?

Yeah, great question. So, uh, In terms of the history, actually, so Factory, we were started in April of 2023. So this was basically 5 months after ChatGPT came out, which kind of kicked off the, the whole craze in more of like the public eye. Obviously, you know, LLMs had been worked on for a while before then, but at the time, I think in, there was an exercise that me and my co-founder Eno did that I think really, got us hooked on this vision, which was, you know, this new, when it was just ChatGPT that came out, there's this new paradigm of having ChatGPT like kind of on one panel and then your IDE in the other. And there's a fun game that we played, which was basically how much of the code that you wanted to get written, could you get ChatGPT to output and you're not allowed to write any code. So basically you have to play the orchestrator of like, you can copy and paste stuff and you can say, no, don't do this. Do that? How much of it, you know, could it actually write? And it's pretty interesting that we found that even then with, you know, just base ChatGPT, oftentimes if you could cultivate the right context, subdivide the task appropriately, and then guide the agent, or I guess at that point, the LLM, generally you could get most tasks done. And so it kind of came to us of like, okay, well, given that LLMs are only getting better, if we can do those things, if we can do the context allocation, if we can do the task subdividing and the kind of agentic orchestration or the framework around the LLMs themselves, then you can automate a lot of tasks in software development. And in particular, the things that developers don't always like doing. Um, and so even back in April of 2023, we set out on our mission, which has not changed to this day, which is to bring autonomy to software engineering. Um, and it's, it's been pretty interesting over the course of just over 2 years, seeing how, you know, lots of things have changed. Models have gotten more agentic. The space has gotten much more crowded than it was when we started, but that kind of core vision and value prop remains very unchanged, which is this broader trend of developers going from, you know, being involved in every line of code to a higher level of abstraction, which is now kind of what brings us to something today, you know, the agent-driven development, which I'm sure we'll talk more about. But that's kind of the, the early days and how we got started on that front.

That's very helpful framing. And I like the, the game, if you call it that, that you played, which is what can we get ChatGPT to do? And we assume the models will get better and better over time. Out of curiosity, what was it not doing well?

Yeah, I mean, the most common failure modes were it would fail because you didn't give it the right context or you didn't give it the right specification of like, hey, I actually want you to do this. Um, or the task was too large. So it was like, there's kind of too many reasoning steps. Like, as an example, if you were at the time in 2023, you were asked ChatGPT like, build me a full app. It would just be too much. It wouldn't be able to do all of that. Whereas if you were to subdivide it, you know, chances are it actually would have the capability to do that. So those were some of the most common failure modes. Now dividing those out. So suppose you gave it the right context and sufficiently sized task and gave it the right framework. It still would, you know, there still would be cases where it fails. Um, and those cases where it would fail, oftentimes that would be like, again, in that time period, uh, in like early 2023. That would be because it didn't have the tools that we as humans have at our disposal to actually verify if the code that you're writing works. So, you know, at the time you couldn't really set it up to run tests. Whereas now the agents, the droids that we are building at Factory, they have access to full environments so they can go and debug themselves.

You sort of answered one part of the direction, which is why agents and why focus on orchestration versus, you know, building an IDE. Why did you decide to focus on enterprise? Because I'm sure a lot of folks can get leverage out of orchestrating agents. I could do that in my role as a VC. So why enterprises?

Yeah. So I think, you know, the reality is, is, and this was something that was maybe not intuitive at first because, you know, I live in San Francisco. Most of my friends are working on really cool startups, working on really like frontier engineering work or research and all that. But the reality is, is that's a very, uh, non-representative sample of engineers. In the United States, let alone the world. And I think the promise of AI and AI agents is increasing the leverage of every engineer, every person, but in this category in particular, every engineer. And I think focusing on just the engineers who are always going to look at the coolest frontier new tools, it's kind of doing a disservice because there are orgs that have 50,000 engineers where maybe 49,000 of them don't really care that much and aren't staying up to date on every new tool that releases and every, you know, weekly, like all these updates, like some of them are, you know, engineering is their job and they care about it, but they're not, you know, the most AI native. And I think for us, reaching them is key to unlocking this, this promise of leverage that AI and agents for software development, um, you know, we'll be able to do. I think it's also, you know, just from a business perspective, I think it's an important focus area because if you look at the profiles of the people who generally are working in AI code generation, generally it's like, you know, hotshot Ivy League or Stanford grads who want to like spin GPUs and do cool things like fine tuning and RL and all that stuff. And if you talk to them about, you behavior change or like, you know, driving adoption in a large enterprise. It's like, oh, like that's not cool. Like I want to do the GPU stuff. Like I don't want to deal with, you know, teaching people who are less AI native how to use these tools. But I think that, that kind of unsexiness of behavior change from our perspective, that is like huge, huge opportunity to actually bring this AI revolution, not just within San Francisco, but to the rest of the world where they aren't as AI native.

Okay. So let's talk about what it actually takes to get into the enterprise. You mentioned there's a little bit of this like unsexy behavior change problem. So there's that. I'm curious from an education perspective what the biggest challenge is, but also from a technical perspective. I mean, you're dealing with old repos, code migrations, and several challenges. So yeah, how do you think about the behavior challenge? And then how do you think about the, the, the technical challenge?

Yeah, great question. So I guess some of the ugliness of their like engineering environment is actually good to a certain extent because the worse and the more hairy it is, the more eager they'd be to have a solution to it. So on one hand, that helps. On the other hand, for our engineering team, that means there's a lot more work to actually, you know, make sure that factory can go in and deploy and actually understand comprehensively that, uh, engineering system. Um, so there's like two sides of the coin there. So generally it helps, like oftentimes with the enterprise, one of the use cases that we'll enter with will be really nasty migrations that they've been putting off for years or, you know, some large refactors that is planned to take 8 months and a ton of engineers and a lot of, you know, work required on the operational side to even communicate about it. Those are cases where at least the promise of agents is sufficiently positive that even people who might be skeptics are like, you know what, fine. If this can make it so I don't need to do a migration, I'll, I'll, I'll, you know, buckle up and learn how to use these AI tools.

Anyone who's dealt with a migration would agree.

Yeah. I mean, it's a pain. No engineers enjoy doing it. It's also something that's very much welcomed. It's like, I don't think there's any engineer who would be like, oh no, We're now going to have AI doing all our migrations. Like generally, I think that's a, that is a better world for everyone. Um, so that's one place that we start, but then there's also, you know, the behavior change and adoption across the org for things that aren't just these migrations. And that's why for us, we typically start with those particularly painful, uh, deliverables to then kind of build up a reputation in the org and a trust in the org so that even if that engineering org is not the most AI forward. They can point to this and be like, oh my God, did you hear about that crazy migration that we did with Factory? And it only took us 1 week instead of 4 months. And it's like, wait, okay, let me actually try and go and do that elsewhere. Because at the end of the day, developers are, they're very clear and astute at figuring out what tools will make them more productive. And their trust is very, hard fought, you know, if you, but if you can earn that trust and if they can see this is actually moving the needle, then they'll, they'll be very quick to adopt. And so I think, uh, you know, navigating that in these orgs that might be tens of thousands of people, um, it ends up being a, uh, a pretty interesting challenge that requires almost as table stakes the best product on the market when it comes to software development agents.

It's an interesting thinking about migration as a wedge, because when I think about migrations, I think of them as these one-off enormous projects that I've dealt with migrations before. They're, they're, they're, they're not fun, but then you do them. How do you make that transition between— some of it is awareness, but it's very different to use a tool in your day-to-day or weekly and get that embedded as part of your workflow versus to handle a project like a migration with AI. So what does that transition look like? And then what do you need to build so that this becomes a habit for developers in the enterprise to use?

Yeah, that's a, that's a, that's a great question. I mean, I think an important principle for us in how we're building this is, um, this concept of ubiquity where basically we are asking a big thing of developers, which is over time to change their behavior to this agent-driven development paradigm. And maybe, maybe I can take a step back and actually explain a little bit about that. So, that agent-driven development paradigm is basically, you know, we're going from a world where developers write 100% of their code to a world where developers will write 0% of their code. And so what used to be kind of the go-to interaction is like, how do I write this line of code to make this function work how I want it to, to make this, um, you know, system work in the way that I, that I'm intending, like those are kind of the primitives to a new set of primitives, which is how can I scope out a task? And define the validation criteria to know that the task was done well, and then delegate that to an agent as quickly as possible? And then how do I get multiple agents working in parallel? So you're going from this world of like single-threadedness to multi-threaded or parallelization. And, you know, on the way, we're kind of slowly working our way on that journey. Like one thing there is, you know, introducing AI into the IDE.

And that's your belief in the limit that software engineers will be writing 0% of code, or maybe the abstractions look different and— Sure.

Like asymptotically. Yes.

And verification.

Yeah. And I think there are certain, uh, subsectors where this will happen, uh, slower and certain areas where it'll happen faster, but like asymptotically the amount of code that is written by humans, uh, as a percentage is going to be way, way lower. Now the human engineer or whatever the term ends up being called will still be essential in this process, but instead of as an implementer, as an orchestrator. And they will be orchestrating these agents and saying, hey, here's our broader— first of all, what actually broader features or product decisions do we actually want to make? In a world where we have a million agents that we can call at any time, and a key decision is which ones do we choose to actually do? You can do all of them, but generally, as many products can attest, having all the features isn't always the best. Sometimes you want that simplicity. And so decision-making criteria there is going to be extremely important and will be human-owned. Scoping out how to actually implement certain things will be important. So you can define goals and success for these agents. And you also need to give them the ability to empirically validate it. So give them the ability to run tests or to spin up a UI and verify functionally certain things are working. So the better that you as a human engineer aren't making those decisions and then kind of scoping it out and delegating it to agents. Instead of this world in the IDEs where like the AI is making you 10, 20, 30% faster, you'll have 10 agents, 20 agents, 30 agents working in parallel. So you are 10x, 20x, 30x faster. And I think it's a, it's a fundamentally kind of different interaction pattern. And so with that in mind, we need to meet developers where they are today. And where are developers today? Well, sometimes they're in their terminal, sometimes they're in the IDE, sometimes they're in Slack. Sometimes they're in Jira or Linear. And we need to make it such that they can engage these agents and we call our agents droids. They can engage these droids wherever it is that they're working because, you know, lightning might strike in their head of like, oh wait, this is a perfect task I want to delegate. And just like a human, you might ping someone on Slack being like, hey, can you go do this? Or you might see a ticket on Linear and be tagged someone and be like, hey, wait, can you go do a first pass implementation here? Um, so it's important to be ubiquitous such that as they're like, as developers are learning this new interaction pattern, it's easy for them to just like, okay, great, let me go do a delegation.

So they see the proof in this like really difficult thorny thing, like the migration, but then whether they're in Jira, whether they're in their IDE, like they have some way to connect to and use Factory easily. So it doesn't feel like I need to adopt an entirely new workflow.

Yeah, it's like, oh wait, I want to do that. I have to go all the way over here to go, you know, like imagine if there was a new hire on your team and the only way you could interact with them was on like some separate messaging platform or something. That'd be very inconvenient. Yeah. Yeah. You want to look wherever a human engineer might be. You want to make sure that you can delegate to a droid. Like if you wanted, you could set it up such that you could literally email a droid to do a task. Now, I don't think any engineer would actually do that, but you know, the principle is that like these should be ubiquitous.

Makes sense. I mean, I'd imagine there are quite a few pieces to that. So some of it is the playing nice with existing tools and workflows, which we talked about. Some of it is understanding what's going on in the codebase, indexing the codebase, being able to retrieve appropriately. Some of it is, to your point, you could have a bunch of different agents, but there's value in simplicity. So I'd love to go piece by piece and understand what you all are doing to help the agents better understand the codebase. And then maybe we can get into some of the different droids. and, and the challenges around building them.

Yeah, absolutely. So, uh, table stakes here, and actually something that we come back to a lot in terms of, so we talked about principles. So one of those principles is ubiquity. It needs to be everywhere. A principle that's very related to this is that we really need to think of these systems and their success and failures similar to human colleagues that we work with. And if you onboard a human colleague and just gave them access to GitHub, let's say, they would not be set up for success because they'd be missing all of your communications on Slack. They would have no context about what sprint you're on or the tickets that are existing or the outages you're seeing in Datadog or whatever, logging and monitoring and all, all these sources of information. Any good engineer needs access to that. And so it would be unfair to assume that a droid could do as good as a human if they don't have access to the same information. So for us, we integrate with the entire engineering system. And loosely, you know, if you've loosely, the more information you provide, the better the performance will be, assuming it is relevant information. So a failure mode could be, you know, if you integrated, um, GitHub and Slack and then there was an outage and your droid didn't provide a good RCA, it could be, well, hey, are you integrated to Sentry? Well, then how could you expect it to have had a good RCA? Because it wasn't able to look at the logs. Um, so giving access is kind of a really important thing to that end that you mentioned of, um, you know, really needing to have that full, full context of the engineering org.

Does surfacing context for agents look different than surfacing context for humans? So you feed all this context into the system.

It's eerily similar in that we also have systems like memory, um, that allow you, allow the, the droids over time to learn the best practices of the organization, understand at a hierarchical level. So at the org level, but also, you know, this team versus that team, or like this pod versus that pod. This repo versus that repo, there might be certain best practices. And as these droids work more and more, they get closer attuned to how does this organization in particular build in this repo? How does this team like to do these things? It's also configurable. So like similarly to a human engineer, if a human's like, oh, hey, you know, I think we build software this way on this repo, you know, at this new team I joined. But if they're wrong, the manager's like, hey, by the way, we don't want to do that. or we used to do that, or like, or we used to do that, but we don't want to do that anymore. Right. So, so there's kind of a, a balance there of like learning from the existing codebase, but then also forward-looking of, you know, maybe the existing codebase had really poor test coverage. You want to make sure your, uh, your droid is not going to be like, okay, great. We don't do test coverage here. Right. Like you want to, you know, maybe, uh, raise the bar.

I mean, memory is a challenge in and of itself. How do you all, uh, balance? Memory and learning with what you said, this configurable piece? Is that something that's prompted by a manager, engineers, et cetera?

On one hand, there is all of our memory is hierarchical because we want to make sure that, uh, we can kind of accommodate like different subsections of the org. It even goes down to the individual level. Um, because you want to, you want to make sure that these are really tailored, like I program differently than you do. Yeah. And just, you know, if I'm helping, uh, or if someone on my team is helping me out versus helping someone else out. They might have different priors in terms of how Matan thinks about certain things versus how someone else does. And so you definitely want, you know, agents to have that as well. You also want to have certain things that are configurable and certain things that are not. So for example, there's some parts of memory where you want to make sure that, you know, the admin can go and say, hey, by the way, like, you know, let's adjust this because we want to shift our org in that direction. But then there are other things, which is like understanding the ground truth of your code and understanding, hey, this piece of code was written because of this customer support ticket. And we had this document outlining our like eng design plan to address that. Like these are things that you don't want to edit because those are kind of ground truth. Um, if that makes sense.

Yeah. Like this was the intent of the code. And so it should do this.

Yes. And this is like almost like historical trajectories of like, these are the facts of what got us to where we are today. And then there's also, here's like the discussion that we're having in the opinions and how we want to move forward, which maybe that part you want to be more configurable.

How do you think about storage in this context?

So I guess storage in the sense of like, where, where do, where does this memory sit? Yeah. So, so the configurable memory we want to have kind of up in, in the face of the user so that it also makes developers more familiar with how LLMs and agents work. Because that's actually another thing that I think is maybe under discussed is that the more developers understand about LLMs and agents, The better they engage with them and the more they get out of them. So it's really in our interest to not hide anything from developers, but in fact, show them as much as we can that's under the hood. So they get a sense of, oh, I see, like, this is how context is managed. These are how tokens are used more efficiently in this way. Um, and so similarly, having them see some of these things like memory and how memory adapts. So like, we'll inform a user if the droid was like, hey, I just added this new piece of information to my memory. And then that gives them the insight of like, wait, actually, like, don't, yeah, yeah, no. Or, or like, wait, oh my God, that's so true. And there's these two other things that I forgot, but I should add into memory as well.

Yeah. I mean, that makes a lot of sense. One of the very first things you said was when you were playing the orchestration game, one of the, one of the pieces that can be missing is just if you don't have the right context. And so this is an element of making sure the developers are in the loop of like the droids have the right context at all times.

100%. 100%.

And then when you think about agent-driven orchestration and having these droids that do different things, how do you, you know, I think when you, uh, were started in 2023, there was definitely a camp that said there's going to be one model to rule them all that will do all of the tasks in, uh, the software development lifecycle. Uh, I don't know what was the most popular camp, but I'm curious how you think about dividing which droid, how many droids should you have? You know, like what is the granularity of a droid? And then, uh, I'm also curious what the most challenging droid to build is of the ones that you've, you've decided to build.

Totally. Yeah, that's a great question. I think the segmentation, like kind of droid by droid is mostly for better user experience. Um, because in general, some of these tasks have different workflows, like Reviewing code is generally a different workflow than writing code, which is a different workflow from doing a root cause analysis of an outage. Now, a very AI-native developer would be able to do any of these with kind of a general purpose agent and could kind of guide it the right way. But we also want to optimize for kind of ease of workflow and for automation. Like, if you want an RCA to automatically be generated for every Sentry issue, like you actually want to have kind of a predetermined workflow that the agent will go through. Um, and so I guess, yeah, I would say it's more of a UX consideration as opposed to a capability consideration because, um, for the most part, the performance, if you guide an agent in the right way, if you guide a droid in the right way, you can get, you know, strong performance either way. If you're going to set up an automation where you want to, you know, this droid to always be engaged whenever, you know, this thing happens, you do want to have some guidance on like, hey, here's how we do RCA. Like, here are the things that we want to give you access to because even though LLMs are getting better and better, if you can do something deterministically as opposed to, you know, probabilistically with an LLM, you kind of always want to do the deterministic side. And so if you can add some, some deterministic guardrails, it ends up helping on the, on the reliability side of things.

Well, I'm, you know, one of the things that you mentioned is that when you go into enterprises, there is an element of handholding and working with the codebase. So I'm curious when you, you talk about it at the Droid level, how much of that is bespoke per organization versus how much of that, you know, are there roughly the same workflows and, and you see the same things across different orgs?

Obviously software development is very different org by org, but the general processes are very similar. Like in general, doing an RCA, like you need to look at the logs. You need to understand which code, uh, which like, where in the code base, uh, was the, the relevant lines that, you know, led to this, to this issue. Um, when was this change implemented into production? What was the reason that we put that in? Um, these are things that whatever tools you use, you're generally going to have these loose steps. Now there might be an org that we work with where, uh, we haven't built an integration for their monitoring tool yet, in which case, you know, we'll go and help tool that up for them. And generally kind of going through the motion with some of their like kind of engineering leaders is helpful so that they get the right mental model of like how to think through these things. But generally it's pretty, once we show the first few people within an org, it's generally pretty self-starting in that they're like, oh, okay, I got it. And they can kind of take it and run with it. And yeah, and so some, some of the larger enterprises that we work with will, will kind of jointly work together on some custom workflows or like custom droids for some specific use case that they have just to get even more optimization or automation in there. Um, but yeah, does that answer that?

Yeah. Yeah, it does. And you know, you have these different models and you have different droids. Is there one that is, uh, you would consider to be the most challenging, uh, workflow?

Um, I think the one that's the, probably the gets the most attention for natural reason and is also the most general is the CodeDroid because the purpose there is just the coding part itself, which, uh, you know, obviously it's extremely open-ended. You know, that's the one where we'll have people on our self-service plan who have never been developers before go in and use it. And then we'll have people who are doing like kernel optimization use the CodeDroid. And these are, you know, on one hand, they're, you know, build me a website or build me a UI to do something. And then on the other hand, it's like, optimizing something for like high performance compute, which are very, very different considerations. And so, you know, making an agentic framework such that it can kind of cater to both is interesting. But I think part of this kind of comes back to that first principles of like, at the end of the day, if you have an, if you give an engineer, really good engineer access to all of your engineering system, they'll be good at mostly any task. Um, it just is the question of like how immediate they get down to it, or maybe it takes them like 2 hours because they didn't know this certain detail that you didn't explicitly mention.

That makes sense. Is there a part of the SDLC that you think is like, will eventually, uh, be agentic, but just is not there yet today? So like we talked about existing challenge being the generalizability of code, but I'd imagine you know, I'm not with enterprises every day, but integration testing, uh, is probably more challenging than unit tests, et cetera.

Yeah. So that's definitely true. I think one thing that's kind of an— this is slightly going around your question, but I think is going to be super relevant is in this world where you have 100 droids working on tasks in parallel for you, eventually you're going to go from unit tests to integration tests and end-to-end tests. And these systems are better the more feedback loops they can do. And generally feedback loops for like unit tests aren't that expensive. Feedback loops for, you know, building out whatever it is that, that, uh, you know, build, like running a build on whatever software you're working on, that can get really expensive very quickly. And if you have 100 agents that are running build all the time to go and do some end-to-end test, your LLM bill might be, you know, comparable to, to like your CI bill. Um, and so I think that kind of reemphasize that it might put a new, um, pressure on having really cost-effective ways of like deploying changes, um, and running some of these tests because, uh, otherwise, yeah, it might be a new bottleneck on the, on the cost side.

Yeah, that makes sense to me. I think there are a lot of things where as the world change, the, the, the points of pain change and the, the commensurate costs Costs can change too.

Yeah. And it'll also put a new emphasis or new ROI really on having good monitoring and observability because asymptotically you want to get to a world where if you give an agent a task and it has some set of tests, you want to make sure if that set of tests is comprehensive enough such that you don't even need to review the code and you could ship it to production. Now, any engineer would be like, oh my God, I would never do that. But if we got to a world where you could actually trust like whatever, this suite of tests, and if all of them pass, then I am ready to, like, we can shift this into production. That would mean that you can really, like, your parallelization of these agents is just truly complete, like blasting open the pipeline of that SDLC. Yeah.

I mean, it would set to your earlier point, like a level of determinism that does not exist today in LLM coding.

Yes. But it gives you kind of this, um, it gives you this huge ROI of having better instrumentation set up on your monitoring and testing and observability because the more you can trust those tests, the less you have to do in the way of review, which means you can have more of these agents in parallel and your life is not just going to be reviewing code all day.

Yeah. Well, I guess it's good news. Uh, the last guest was Olivier from Datadog and it's also good news for you because you, you're, um, uh, set up with all the observability systems.

That's right. Yeah. Yeah.

And so you get that feedback directly to the, to the agentic system. Is there anything that, you know, we talked about some of the challenges in enterprise. Is there anything that really surprised you or I'll phrase it differently. What was the wildest use case that you've come across with real customers who are using the droids in production?

I think the one, so a couple of things that were surprising. So one, PMs love Factory. I think the idea there is just they don't have a software development workflow. So they're kind of like fully greenfield. And so they're actually very willing to go and adopt this agent-driven development because they're not replacing anything. It's kind of net new. So that's been a surprise is like PMs just jump in all the time. Most of the orgs that we work with, they've now kind of raised the bar on PM deliverables where you no longer can just have like a product spec. It's like you need a working demo. And if they don't, it's like, just go use Factory and go do it. Like, are you too lazy to do that? Like, that's kind of the new bar that's been set, which was interesting.

I've seen this for what it's worth, Matan, even more upstream of PMs. So like when I, when I worked in product, I'd often talk to solutions archi— of course I'd talk to customers too, but like solutions architects who are in the field with customers. And now I've seen a subset of folks where someone talks to a customer and converts their call notes to like a PRD, uh, directly too. And so, uh, yeah, I think AI is cutting out a lot of this kind of conversation, transfer information.

And it's also kind of smearing some roles where before it was like, I'm in my silo and now I give it to you because you're in your silo. And it's instead of like, wait, you can kind of like PMs can do some of the engineering stuff. Or like people on ops can actually instrument some of these like automations that might before have been like, oh, you need an engineer to go, to go to build that. I think another thing that's been a surprise, and this is way more in like the large enterprise side of things, but there have been people who have been using Factory to basically audit their codebase to verify that they're like GDPR compliant, which is very unexpected, very unexpected, but it makes a ton of sense because if you have a super complicated system, doing, uh, some of that auditing takes a very, very long time. Um, and having the scale of like hundreds of— the effective like manpower of hundreds of engineers going in and checking things, um, is, is pretty cool.

That's, that's a very interesting use case that I would not have thought of as someone who's both worked in product and had to deal with GDPR.

Yeah. Yeah.

Uh, I think the second one, uh, I think, I think they both would have been saviors at different moments in time.

Yeah, it was also hilarious to me because the, the first person that brought this up, brought it up as if it was like the most obvious use case. I was like, oh, of course. So we can use this for, you know, auditing to make sure that we comply with GDPR. And I was just like, kind of a little bit shocking, but it's like, wait, that actually, yeah, no, it, uh, it's worked quite well.

So, I mean, on the PM side, how do you think about who you're building for? So, you know, to your point, if in the asymptote software development goes to 0% writing code, looks a lot more like planning tasks, orchestration, et cetera, potentially the skillset of folks who would be comfortable with factory also expands. So how do you think about that? And I'll add another layer to it, which is something that I see anecdotally, is that a lot of the senior engineers are getting the, the 10x boost from AI code gen tools. And there can, depends of course on the engineer and many other things, but between senior and junior and where are you seeing the leverage on teams?

Yeah, so I actually, I would be reluctant to say it's, you know, for this profile or that profile because, you know, on one hand you could say, you know, a senior engineer maybe is more familiar with like delegating and, you know, uh, kind of dividing up or designing tasks or architecting things. But then on the other hand, junior developers are less ingrained in their workflows. So they're more open to being like, you know what, actually I'm going to rethink how I build software entirely. So there are kind of pros and cons. I think loosely though, the best engineers have always been the best systems thinkers. And in this new era of agent-driven development, the best engineers will still be the best systems thinkers. It just used to be kind of a reality, which is great. If you're the best engineer, you know, you can plan and scope things out the best. You also have to go and implement it. Now it'll be such that you still have to do the scoping. You just might not have to do the implementation. Because you can now delegate that to an agent. But, and kind of similarly for PMs, like I think the best PMs and the best engineers often have very similar traits. And oftentimes the differences mostly lie in, did you study computer science and get fluent in this language or not? And I think that's actually something that is somewhat independent from your ability to like think around systems and taste and thinking about the constraints from the user perspective and the business perspective and market perspective. And I think, um, yeah, so I think people, people that are the best at systems thinking.

I was going to say you studied physics.

Very unrel— yeah, like in, you know, a priori you'd say that that's very unrelated to software development, which to a certain extent it is.

I would not say that, but one might say that.

Yes, yes, yes. Um, But at the end of the day, yeah, it's thinking about, uh, very multivariate problems that have a lot of competing constraints and trying to figure out what is the optimal solution that accounts for those things. I think that is like the general principle that is relevant to all of these, um, disciplines.

For you personally though, does that change how you think about building Factory? So, you know, you want to build a tool that works for PMs and works for engineers, works for junior engineers, or work for, works for engineers across all levels.

Roles.

Um, how does that— how do you think about prioritizing? You know, one of the things that we talked about earlier was meeting folks where they are, being able to integrate with existing workflows. How do you prioritize across that pretty diverse set?

Yeah, so squarely we are building factory for engineers, but I think what's going to happen is this smearing between these different labels of roles. And it's, you know, I don't know, who knows what it'll be called, maybe orchestrator, builder, whatever it might be. But we are starting with engineers, but the point is, 3 years from now, the people that are doing this agent-driven development might be called engineers, they might be called something else, but that is kind of the, the persona that we are building for. In terms of how we're building Factory as well, I think there's a funny note, which is, this is the first job that I've ever had. You know, I was a theoretical physicist for my whole life just doing research. And then my co-founder, Eno, he was at Microsoft and then Hugging Face, which are maybe 2 of the most different. Engineering orgs and cultures there is. And so, you know, we like to say that we have full coverage of like the most bureaucratic org, the least bureaucratic org, and then no org. And so we kind of have a coverage of that to make sure we have all, all perspectives on, you know, thinking about building an org, but then also, you know, engineering itself.

How about on the, you know, we've talked a lot about building for enterprise. So you build for all these different personas in the enterprise. Uh, we talked about droids and indexing codebases. What about the actual guardrails? Like, to your point on folks prefer determinism, I think that's especially true in enterprise. Uh, what are enterprises asking of you, uh, to build, uh, to gain trust?

Yeah, so I think a really important thing here is agentic systems are very cool, but they also have very, uh, large potential failure modes, in particular in the enterprise. Like, you know, there are some tools out there that have features called YOLO mode. And I think YOLO mode is not something that's particularly compatible with the enterprise. And I think it's funny and like—

Maybe for the PMs if they're not touching the codebase, they can go into YOLO mode.

Yes, but like, and I think it's funny to laugh about, but there will very likely be some billion-dollar incident in the next few years that is caused by an agentic system that has too much access. To something and goes haywire. And I think for us, this is something that we've had in mind since day one, which is we need to provide these enterprises with very clear control and governance and auditability on how these agents are going to work in their codebase. Are there certain commands that they're, you know, they always need explicit permission to run? Are there certain parts of the codebase that they can't access? I think having clear control on this is it's basically just as important as when you onboard again, a human engineer, do you suddenly give them access to every codebase, the ability to push to prod without any like review or approvals? Like, no, like you need to set up.

Depends on the company.

Right. But yeah, yeah, fair. Or like, uh, you know, like, do you give them like AWS admin privileges from day one? Like probably not. Right. So, um, but at the end of the day, you want to build that trust with this new person that you bring on your team and you eventually want them to be able to do some of these higher leverage things, but, uh, you need to have that, you know, trust but verify, um, ability.

Got it. So that makes sense on the guardrail side. So we have the orchestration, there are enterprise guardrails around it, and then there's memory so that the, the, uh, the agents keep getting the agent, agentic system keeps improving and getting better. Um, you're using, from what I understand, and correct me if I'm wrong, you're using models out of the box?

That's right. We're, and we're fully model agnostic. So we don't, developers can pick whichever, whichever model it is that they, that they want.

Is there something, there's been a huge push towards better code gen in models today, towards reasoning. Is there something missing at the model layer today that would unlock the next chapter for you in Factory? Is there something where, you know, if you could wave a magic wand and get it from the model layer, it would, it would be very helpful?

I mean, I think the model providers are doing a great job of making these models get better and better at almost accelerating pace. And I think this has been said before, but really there's still, if the model stopped today, there's still so much ROI to be delivered and so much of software development to like kind of increase. There's so much leverage to provide these developers for where the models are now. And so almost like, I feel like it would be greedy to say, oh, you know, these need to be even better.

Now that said, well, I'm letting you be greedy here. It's okay.

Yeah. Like the directions that I think will be more exciting are, you know, better context management and like real context, not, you know, there, there are some models out there that are like, oh, you know, infinite context or arbitrarily large context, but generally those have a lot of loss in terms of the context that you bring in. 'cause there's kind of only so much that a model can focus on in a given window. But truly increasing their capacity to take in information means that you can subdivide problems into larger chunks. Like you kind of need to break it down less, which makes it easier to improve the success there. And I think also, for us, when these model providers get access to more niche languages or proprietary languages. That just makes it easier for us so that we can go to whatever enterprise, um, you know, might be still using that niche language and we can actually help them do, do that migration. Because right now there is a lot of the, a lot of the lift on kind of base knowledge about languages resides within the models and not within the factory itself.

Out of curiosity, 'cause we talked about starting with migrations, is that still the typical wedge?

Generally, yes. Generally it's, I mean, because everyone has that thing in the backlog that's been there for years that they're dreading doing. Yeah.

Because I was going to say, from a go-to-market perspective, uh, how challenging is it to identify, okay, you know, this is, this is an organization that is ready to invest resources into migration today. It's not something that's advertised or, uh, you go on LinkedIn.

Yes, but it's almost like you can, you can almost guess. Just by like almost, like almost by the vibe really of like, oh, this company's been around for a while. You talk to the right person and you can basically like see it in their eyes when you mention migration. There's, you can like see they're thinking about, oh man, they're like that, that behemoth that's like sitting in Jira that no one has touched. Yeah.

You have incredibly competent agents. What does the world look like for, you know, app layer companies, enterprises, et cetera? When the agents become incredibly competent.

Yeah, this is something that I think is really exciting, which is like, basically, we as humanity have only seen this software that it takes at the limit, like about 100,000 engineers working together to build. And that might be like, you know, some of these cloud providers, let's say. Like, we have never, we can't really conceive of—

And they're not truly coordinated, right?

Like, there's so much lossiness. Yes.

In large organizations.

Totally. Now, a question is like, what is a, what is a, a type of software that we could only build if we had the equivalent of 1 million extremely capable engineers working together on? It's kind of hard to even conceive of. To a similar extent that some of the software that we build now, if you were to tell someone in the '80s about it, it's like, no, they're like, they wouldn't even conceive of the type of complexity that we could, that we could deal with. Um, and so I think basically the table stakes for every org is going to dramatically increase. Um, I'm very much not a believer in this idea that, oh, you know, the number of engineers is going to go down because take two competitive companies. If one of them, you know, cuts headcount by 10, because now each engineer is 10x more productive. Well, this other company that didn't cut that headcount is, you know, way out, outshipping this, this competitor. Um, And I think at the end of the day, there's a plethora of problems in the world to solve. And generally software has gone and plucked like the lowest hanging fruit. And now we're just kind of going higher up the tree to get more fruit. And I think it is so clear that there are so many, there's still so many problems in the world that, you know, having more leverage for each individual developer will now be able to solve even more of them.

I mean, I think. No one knows what it will look like at the limit with a million engineers. You pose a very interesting thought experiment and question. Um, but I am curious how you think about the architecture today, the, the, the, the architecture of your system and how that evolves with scale. It may not need to be a million, but, uh, when you think about orchestrating and routing and the cognitive architecture of your system, What, if anything, needs to change as, as these numbers scale up?

Totally. I mean, it just puts more and more emphasis on really clear monitoring, observability, and structure. It's kind of similar to, to a human organization. Like if you're a startup with 10 people, you don't really need that much like rigorous rules and process and all that because it's like, whatever, you're 10 people in a room, you can figure it out. If you're an org of 10,000 people, if you're 10,000 people in a room, is going to be absolute mayhem and you're probably not going to be very productive. And I think similarly, it's just having the equivalent of a million engineers, you will get insane ROI if you are really precise and thoughtful about how to structure the different mechanisms and processes of decision-making and building and even like the software development lifecycle itself and determining when things go into production, how you're triaging certain issues. So I think while this, like this kind of blows open part of the pipeline, there might be other bottlenecks that emerge if you don't also solve that scaling problem, which to some extent is like an operational question of how do you, when you have so much bandwidth, how do you kind of allocate and make sure it's all kind of going through that, that development lifecycle, um, in a way that doesn't break everything and doesn't like contradict itself.

Yeah, I guess one way to think about it is what does an optimal org chart look like for, for droids? And then like incorporated in that org chart is a set of orchestrators. And there's a question around what does their collaboration look like? Where at, you know, at which point do humans begin and the agents or humans end and the agents begin and vice versa.

Totally. Totally. And it gets, it's like extremely exponential in its complexity because for each, you know, additional kind of actor or agent that is introduced into this problem, the number of now new relationships that emerge grows very dramatically. So, you know, if you're a 10,000-person org and you introduce another person, there are now 10,000 relationships. Like in this graph of connections, like each incremental person you add is going to be kind of a huge layer of complexity. And so it puts that much more emphasis on, um, process and organization, um, and clarity on that, which I, I, I feel like that is a little bit of a non-answer just because it's like, there's so much unknown, but this is part of like, even in, in, as companies have gotten larger, like the theory of company building is being developed now over time as companies are becoming so large.

Well, how do you think about these architectures today? Is it largely a function of you get feedback from working closely with the enterprise and you're using factory in your day-to-day and there is a level of intuition or?

Yeah. So, so I mean, we're, we're of the privilege, like we're, our team is still pretty small, like we're less than 50 people. Um, but in terms of some of the processes, I think one principle that's very clear is everything needs to be written or recorded. Because if we want our agents, if we want our droids to make decisions as good as our decisions, they should have access to the same information. So if we're having an engineering sync, that needs to be recorded and the droids need to have access to it. If we're meeting with a customer, it needs to be recorded. We need to have a document about, you know, our thought process and our discussion there. Slack conversations. Droids need to have access to all of this. There can be nothing that's like tribal knowledge or that only exists in people's heads, because that now means you have a discrepancy between what your agents can access and what your humans are aware of.

That makes sense. I think it speaks to the blending of roles that we talked about earlier, like talk to customers, straight to PRD, straight to, straight to like a V0 prototype. And so the agent, by getting, by being in the meetings, can get all of that. And then how do you think about recruiting? Just, you know, I think that finding folks who both are strong engineers, but interested in a bit of an embedded model as well as you discussed. So you have to go into the enterprises. Understand their hairy issues? How do you recruit for people who have that joint skillset?

Yeah, I mean, I, that's a great question. I think we are talking about this smearing of roles. People that we bring in need to be very obsessed with this software development itself, developer, whether across the board, whether it's go-to-market, product, ops, engineering, everyone needs to be obsessed with the, like the process of software development. Have very strong empathy with the developer profile, really care about the craft of software development and how that is changing. And, you know, have a necessity to see customers in how they use our product, use our product day to day. I think for us, you know, one of our values that we like is, you know, there's the famous Jeff Bezos value, which is customer obsession, right? For us, we kind of, uh, had a little bit of a twist on that, which is create obsessed customers. We need to build something that's so good. It is not enough for us to be obsessed with them because that can be a one-way relationship. We want to make something that is so good that they become obsessed with Factory and the platform itself. Um, and I think people that are aligned with that are pretty rare because I think it's a level of, a level of obsession that is very like definitionally not normal. But I think that's what's required to, to bring about this, this behavior change.

Yeah. And I think those are very related, at least in one way. You know, if you want the customer to be obsessed with you, you have to be obsessed with the customer. Speaking of your, your org, I spoke to Fran who told me that you wear the same thing every day and you eat burritos for every meal. First, confirm or deny?

Confirm. This is actually a special day. Normally I have a black t-shirt under this. Normally it's just that I'm wearing a sweater today because, you know, you dressed up for the podcast. Yeah, exactly right. Yeah. Yes, I do eat the same meal.

Yeah. I was gonna ask, how have you changed the most as a person and a leader since you founded Factory? Since we know about your consistencies now.

Yes. So still eating the same thing for breakfast, lunch, and dinner every day since starting Factory.

It's a balanced meal.

Oh yeah. Yeah. It's about, it's, you know, a breakfast burrito, a salad, and then chicken katsu curry for dinner. Anyway. On a more serious note, I think the things that have changed, so I joked before how this is really the first job I've ever had, you know, prior as a physicist. So I was just spending all my time alone. I think Factory gave me such an appreciation for the fact that everything comes from the team. Everything doesn't matter how good your vision is, how fancy your investors, how skilled you individually does not matter. If you do not have the team, you will not build the right product. You don't build the right product, you will not get the customers. It's like, it all stems from the people that you work with. And it's so self-fulfilling because if you are working with the people that are the most ambitious and the most relentless and aligned, then it kind of, on the one-off day that you might be feeling a little bit less, like being around them kind of brings you back and it's intoxicating. And I think that is something that I had heard, but I did not realize viscerally how true that was. And so that's why I still, to this day, I think I spend most of my time, if not, you know, with the team, it's on hiring and bringing new people that I think would be a good fit. Um, so I think that's the kind of biggest change on a more serious note. I think on a funnier note, I switched from coffee to Celsius, uh, which has been a, a fun change, maybe not as fun for my, uh, health, but you know, how and when did that happen? About a year ago we started. Bringing Celsius into the office and I was like, oh, like what's this? I'm seeing this everywhere. I tried it and it's, uh, it's good. It's got a lot of caffeine. It, it, uh, it works quite well.

I love it. So it all comes back to spending time in the office either with amazing people or with the Celsius.

That's right. That's right. All right. Preferably both.

Preferably both. Matan, thank you so much. Thanks for coming on and excited to see what the world looks like when, when agents are ubiquitous.

Truly.

Yes.

Likewise. Thank you so much for having me.