An unfiltered conversation with Matan Grinberg, CEO of Factory AI

People are elevated who have the skills that have kind of made for the best builders and developers in the first place, which has always been systems thinking and thinking around constraints. The reality is that even as models get better, there will be certain tasks that are kind of within range of being delegated and autonomously solved. And then there will be some that you, you as the human developer, still want to be able to kind of monitor and jump in as necessary. A lot of these tools, it's like they're super neck and neck and it's like, well, this person did this and we're going to lose half our users if we don't do that in the next payment cycle. So we have to do that, right?

I'm super excited to have this conversation. We haven't caught up in a long time. I'm happy to catch up. For folks who don't know what you're building, do you want to give just like a really quick intro of what Factory is trying to build and the problem that y'all are solving?

Yeah, absolutely. Great to see you again and great to meet you, Nolan. So at Factory, our mission is to bring autonomy to software engineering. And what that means more concretely, we've built the home for agentic software development. And what does that mean more concretely? Well, maybe stepping back, talking a little bit higher level, I think everyone knows that software development is going to change dramatically over the next few years. But so far, a lot of the AI coding tools haven't really transformed behavior that much. In general, the tools we use look roughly the same with some new features here and there. And really our take is that indeed software development is going to change dramatically. But in order to actualize that change, developer behavior needs to change. And it needs to go from kind of the typical software development lifecycle that we've seen over the last like, 10, 15 years into one that's agent-first, where really the kind of focus where human developers spend their time shifts from coding to really understanding and planning and systems thinking, thinking about the constraints from the business, from the product, from the customer, and then translating that into clear plans and instructions for agentic systems to then, you know, you delegate it to them and then they go about executing it, you might iterate back and forth, you might jump in, but really kind of stepping into that more orchestrator role as the developer. And that's the platform that we were building really from scratch. So not kind of going iteratively from the existing interaction patterns like the IDE, but instead rethinking it from the ground up. And that is what the Factory Platform is. The autonomous systems that we have that you delegate these tasks to, we call them droids. And so they can be task-specific. So things like RCA, incident response, as well as just general coding, feature building, PRD creation. Yeah.

I love that. I feel like one, the Droids brand is sick. Like, I think you need like good Droid swag. I feel like that goes really well. How do you think about this? Like what you just described to me sounds like some tasks that are done by software engineers today, but like the sort of pool of people who might be able to do those tasks seems like it's like a much wider audience of people than potentially like traditional software engineering folks. Have you seen already, like you're, you know, a few days away from going GA and having everyone available to use the platform. You obviously, you have people using Factory today. Is the audience of people who are using Factory today like also sort of already widened and like PMs and designers and stuff are sort of tasking droids to do, to do work on their behalf? Or is that like still a little bit more future-facing?

Yeah, that's a, that's a great question. I mean, I think we've seen even over the past few years, some like early evidence of this with, with like low-code and no-code tools, getting people who previously weren't developers interested in the space and excited. I think something that we see is really roles getting a little smeared in like EPD teams where before you might have a PM or a technical PM or an engineering manager or an IC. These are all seen as like pretty distinct. But as AI gets better and as you have, you know, tools that you can delegate tasks to, we kind of zoom back out and people are elevated who have the skills that have kind of made for the best builders and developers in the first place, which has always been systems thinking and thinking around constraints. And really code has kind of been a thing that you have to do once you, you know, go through that systems thinking and once you go through like understanding the constraints that you need to satisfy. Since there was no AI, you have to go and then actually implement it. And the reality today that we're going towards is that, you know, once you actually have a clearly defined specification of what it is that you want to build, you no longer need to actually go in deep and actually implement it. You'll be able to, you know, and now it's not 100%, right? You still, as a, you know, human, need to be able to go in and debug and like understand the, the actual implementation. But we're very rapidly getting to that place where you actually maybe 1% of your day as a developer will be actually going into the code. And what that allows is there might be PMs who never studied computer science. They didn't take the time to become fluent in any of those particular languages, but they're still able to think at that systems level. Tools like this just enable them to not need to gain fluency in that language. So there's definitely the smearing. We also see like, you know, even some like solo developers or friends that we've given access to, there are folks who have never developed in their life who are getting a ton of value. In the enterprise orgs that we work with, which, you know, that really is our focus, the large enterprise, we'll see technical PMs, engineering managers, like even C-suites who haven't coded in like 20 years get involved because it does kind of just lower that barrier to entry and just allows anyone who has that mindset to be able to jump in without some of the barriers that don't necessarily filter for systems thinking or engineering thinking. It's instead just like you either know the language or don't.

Question 2 about the droids, like trying to comprehend. So do I go on to Factory AI? Are there like a bunch of different droids I can pick from that already have kind of pre-built workflows? Am I starting from scratch? Like, how do you actually go in and leverage technology and build it to be beneficial to you?

Yeah, great question. So to start, we kind of, you know, when you jump into Factory, there are 3 that are going to be put up like front and center because these are the ones that as we go GA, we've just seen the most like general purpose applicability for. And that's the Code Droid, the RCA or Incident Droid, and the Knowledge Droid. And so the Code one, pretty self-explanatory. And this is the thing that I think you'll see most people when they first jump in just to see what it can do will be like, spin me up some zero-to-one application or some like build me some feature on some existing code that I have. That's one that's kind of like table stakes just to see like, all right, go generate me some code, run it. I want to see it do the whole agentic loop where, you know, it might make a mistake. It'll see the output from CLI and then iterate based on it. So that's kind of the first thing that most people do just to get to like watch the agent do things without you being super involved. The incident response one is kind of when you get a little bit deeper and, you know, you're actually in some production setting and want to save time instead of needing to go super deep into, you know, your Sentry logs or Datadog or whatever tools you might use. Seeing Factory able to go in and kind of pull in that information from your codebase, from your docs, understanding what the PRDs were in the first place that led to this code change that might have led to some incident. And so that's kind of a little bit deeper because it's actually typically in production. And then the Knowledge Droid is one that takes advantage of a really big focus for us, which is context of the entire engineering system. So similar to how when you onboard a human engineer, you don't just give them access to your codebase and then say, all right, go have fun. Generally, you'll also onboard them to tools like Slack or Jira, Linear, Google Drive, Notion. So the kind of first principles idea for Factory is if you want the autonomous systems perform— to perform as well as a human, you need to give it access to similar information sources. And so the Knowledge Droid kind of shows off that ability to really go deep and understand the interconnectedness of all of this information, knowing that code changes don't just happen on their own, but they're connected to Slack conversations or sprint plans from your Linear or Jira. And then using that, you know, developers, let's say in an onboarding scenario, can go in and get a quick understanding of how the organization is working, what it is that they might want to do for a given day, given project, that sort of thing.

Quick question off that too. For the enterprise customers you're currently engaged with, are they usually leveraging all three of those droids? Are they pulling just one? Like, how does that kind of work? Is there like a full tech stack that you're eventually trying to sell or get in front of everyone?

Yeah. So I mean, generally the way we work with the enterprise is we'll typically start with one use case that's really top of mind. And normally for the large enterprise, there is some beastly migration or version upgrade that they need to get done, in which case it kind of depends on the size and scale. Sometimes they might use multiple droids, sometimes they'll use one specifically for, let's say, like a Java 8 to Java 21 migration. But that's typically where they'll start, especially because, you know, it's outside of the IDE. So it is a net new behavior and you kind of need a fire under the ass to actually motivate that difference in behavior and there's no fire under the ass, like a necessity to do a migration in some, like, let's say 4 months, let's say. And then we come in and say, hey, you know, use Factory, you can get it done in like 2 weeks. That's kind of an easy way for them to say, all right, well, let's try it out. Let's see. Once they get that win, then they can go broader using different droids in other aspects of their development. But generally for the large enterprise, we'll start with one pretty lump sum task before we go broader for just general developer productivity. Yeah. Yeah.

Matan, the way you've described Factory, and as someone who hasn't yet gotten to play around with it, though, if you've got an invite code—

Which we should change. Let me send you a link after this.

I love it. I love it. I'm excited to play around. This angle of the model and the system being more agentic, again, for folks who haven't seen, and we'll have a link somewhere and you can see demos and stuff like that in the show notes, but for folks who haven't seen this, How much more agentic is Factory today than, insert your favorite AI product right now, Cursor, Windsurf, yada, yada, yada? Is it really the much longer running tasks that are happening, or is it a familiar human in the loop after 30 seconds or a minute from a cogeneration standpoint?

Yeah, that's a great question. So I actually, I'm really glad you asked that because I think something that we're really opinionated about is that You actually need both. You can't just have the pure agentic delegation, nor can you have the pure pair programming back and forth. The reality is that even as models get better, there will be certain tasks that are kind of within range of being delegated and autonomously solved. And then there will be some that you as the human developer still want to be able to kind of monitor and jump in as necessary. And so a big thing for us is allowing the developer to kind of dynamically and on the fly adjust whether they're doing more collaboration or more delegation. And so to answer your question more directly, when they're in the full delegation mode, typically what that looks like is, you know, they want to work on a feature within Factory. They'll generally have a plan that they make first. So we're, you know, users of Factory generally learn pretty quickly you don't want to just shoot from the hip. Because it's kind of, you get what you put in. And if you just say, hey, go do this, it'll make assumptions that you didn't explicitly state, which again, kind of goes back to the constraint-based thinking. If you say, go do this, chances are there are constraints that you have in your head that you didn't actually verbalize to the system. And so it'll give you something back and then you'll be upset because it violated those constraints because you didn't actually say them. So, you know, generally within Factory, they'll, you know, come up with this plan, they'll decide, you know, maybe this part I want to do myself or, you know, someone on my team kind of has ownership there. This substep I want to go delegate. What that delegation could look like is Factory generating the code, actually executing the code for, you know, whatever feature, verifying that it passes certain tests, which again, you know, tests would be involved in this plan. If it doesn't pass those tests, it can take in those output logs based on the output logs, understanding the failures, can keep iterating until it hits that acceptance criteria and then proceeds from there. So generally, if you're kind of, you know, want to see the full delegation magic, you as the developer have the control of like how many iterations will it try if the tests are failing, let's say. And if you want, you can really just go like fully hands off and just like kind of give it the keys and just say, okay, keep going until, until you satisfy this. If you'd like and if you're seeing it's kind of going astray, similar to how you would work with a human developer, a friend of yours, you might go and say, hey, by the way, I think you're messing up on this axis. Like, have you considered this? Or maybe let me throw in some docs that you might not have considered that would actually make you realize that you should use some other module, let's say, and that'll solve your problems.

For the enterprise companies you're talking to, are you seeing a friction point with folks wanting to delegate? Like, do they still want that hands-on experience? I feel like obviously the larger organization and the more sensitive data, kind of goes into a tricky situation. But like, I'm just curious as far as how many people are like, all right, let's just see what this— see what happens, right? Let's give it the keys and see how well it drives.

Yeah, yeah. This is a great question. I think generally in the piloting phase, people really want to see that that capability is there. But then when we get to the actual deployment, they want to know that they have clear governance and control over what things can the agent do kind of without permission and what things are like what even like CLI commands. Does it need explicit user permission for? Also having the ownership of any of these changes or commands be clear for the developer so that they kind of can't just put their hands up and be like, oh, something happened. I don't know what went wrong. So in terms of the actual deployments, having like even like blacklisted or whitelisted commands for auto-running, that's another thing that's really important. I think another thing to emphasize here as well is a lot of these tools that have some of these agentic capabilities typically will do so either locally. And so that's, you know, the IDEs that have agentic capabilities. They might run CLI commands locally to test the code, that sort of thing. Or there are others that are more like ticket to PR type agents, which typically spin up cloud instances where they'll run the code in the cloud, verify that it works, and then submit a PR. What's cool about Factory is you can actually do both. So you can have parallel agents running locally and parallel agents running in the cloud. The beauty there is the ones in the cloud typically would be the ones that you're in some sense is more confident on because you might not want to go hands-on and like carry on from wherever it left off. You might just want to, you know, you're confident it'll end in a PR or ones that you're a little concerned about what commands it might run and you want to make sure that it's running in some like remote instance and not in your local environment. On the other hand, the agents that you might want to run locally are ones where it's something that's super complicated. You want it to go and kick the tires for a little bit before you go in yourself. But then this way you don't need to then pull down a branch to start working on it locally. You can just pick up where it left off. But then on the other hand, since it is running locally, you'd want to maybe monitor a little bit more because it could mess up your environment if you're not, you know, sure of what's actually going on.

Quickly off that as well. Sorry, Logan, I'm just curious too, because like for the workflows of it, like, are you training it on a human, like someone who's in that position doing it day in, day out? Like, here's how I want you to actually act. Like, are you actually with those droids that are kind of pre-built already brought that workflow into it? Like, how do you actually make sure it's going and doing everything that your best or top developers doing today?

Yeah, this is a great question. So this is what I think maybe some might consider like the less sexy parts of this because, you know, generally in the AI codegen space, it's all like you want to do the fancy like RL or fine-tuning and that sort of thing. I think for us, you can go a really long way just having explicit documentation of the best practices within the org. And this is something especially for the larger orgs where we really, they have a lot of like codified standard operating procedures. We go about doing in that deployment, making sure that the droid is actually customized to that use case. And what's nice about that is it's actually a lot cheaper and as effective as fine-tuning. And if this exists in like a YAML file, let's say, or some, some doc within their codebase or even some config of theirs, they can also go and manually change it as they decide, oh, actually, you know, our best practice is we want to move it in this direction. So that's been something that's really successful. And I think the important thing, though, for scalability in these large orgs is Making sure we don't have to go and do that every single time, but rather teach them to understand how these systems work. And then kind of they end up being the internal champions that can go and say, oh, hey, this team over here that wants to use a Droid for this task, I know exactly how to set you guys up. And then they're good to go.

Yeah, Matan, you've spent the last year plus building this product from an enterprise perspective. And now that you're making it available to like the general developer audience and obviously like the— and I have a bunch of questions that I'm curious to hear your take on, like the model progress story and all that that's happened in the last year as well in conjunction to the stuff that you've been doing from a product perspective. But how do you think about like solving this problem first for enterprises? Because I like alluding to Nolan's point, like I feel like solving this for enterprise is just so much more difficult. I'm thinking about just like internal code bases at large companies. There's a lot of human complexity, a lot of system complexity. Versus like, you know, the random developer who's just like vibe coding something. And obviously you built this product in a world where like the models were capable of different, sufficiently different things. So like maybe the value creation was only enterprise, but like, I'm curious, like, is that roughly why enterprise first or like why? I'm curious to get that story.

Yeah, great question. So I think part of it is because at the enterprise is where there's the most differentiated value from having a really good retrieval system and like codebase understanding system because kind of Considering the other case of like 0-to-1 vibe coding, most of the alpha there in terms of the quality is from the models, not the product or the UI. Generally, the product or the UI serves a good purpose, which is sometimes if you're working with someone who's never been a developer, you want to make it really welcoming and kind of like easy to use for them. But in terms of the actual, like what differentiates the quality of the output, generally that alpha kind of lives within the models for those 0-to-1 use cases. There are definitely exceptions. To that. But generally that's the case. I think one exception worth calling out is UI can actually get more in terms of the constraints that the user is thinking. So if you have a better UI, you can extract that more easily from the user so that your output maybe is not better quality, but better tuned to the constraints that they implicitly had in their head that maybe a tool with a worse UI or worse kind of like experience for that user wouldn't pull out of them. And so then they might get something that matches their expectations less. But yeah, so the reason, the reason for the enterprise focus is like, obviously there's a lot of alpha in the models themselves, but alone you can't just drop any of these models into one of those codebases and hope something good happens because there's just so much context, so much dead code, so much tribal knowledge kind of stored within the thousands of people within these orgs that the models just won't have access to. And they can try and they, you know, in more local instances, localized changes, they can provide value, but it's just generally pretty underserved because also a lot of that problem of navigating the entire engineering system or even understanding the role-based access control that you need for— there are some repos that only some people have access to or like Google Drive. There are just so many permutations of that. And it's a little less sexy than dealing directly with models and with generating cool apps from scratch. That I think people haven't touched it as much. Um, but that's kind of the type of problem that, you know, the team at Factory, we, we jump on.

So yeah, do you think this will change over time at all? Or like, you, or you still think like, I could imagine a lot of the primitives or the scaffolding that you've built to make it, make this work well for large enterprises would like also potentially mean you could build a differentiated like top-of-funnel vibe coding type of, and maybe that, that is what people will end up doing as part of of the, the GA launch and you'll see more of that. But like, do you, what, what's your expectation going into the launch? Like, is it, will you see a bunch of people doing that?

Yeah. So I think, I think that's a, that's a really good point. I think, um, so first of all, I think the UI that we have is really new and innovative and I think people will really like it. I, I will say, I think the quality is not going to be some like crazy difference in that zero to one use case, because I think in that zero to one use case, most tools perform mostly the same way. Even most of the raw models, Just like if you give it some query, it'll perform mostly the same way. I think the difference is, and a lot of people are kind of aware that, you know, some of these no-code tools are like zero-to-one app building tools. It's kind of like taking out debt. And so you can make a cool app at first, but if you start needing to like make a lot of changes or support a lot more throughput or actually like set it up on its own and, you know, kind of in these cases, like let's say you wanted to actually use this as a piece of software to have a business built on top of. You end up having to pay that debt. And really most of the time you end up needing to get a real engineer in there who can go and kind of like fix all the wires behind the scenes. I think that's where there's going to be a big bifurcation in terms of performance between us and some of the other tools out there that are kind of catered towards that zero-to-one use case. So very excited to see that play out, especially, you know, the weeks following launch when people get to that phase with some of the software that they build.

Yeah, Nolan and I had a long conversation with Eric Simmons from Bolt about generally this exact problem, which is like, how do you, especially for them in the context of like having a much less technical user persona than I think it sounds like you have with Factory. The challenge is like people have great ideas and they take the first step, but like the current systems don't take you the next step of like, you know, how do I do support? How do I do marketing? How do I actually even just deploy this thing and like turn it into something that people can use? So I'm super excited. I really hope you hit this part out of the park because I do think it's like a requirement for this like current era of like everyone being able to create software for that to actually create value for the world. I think people need to be able to do these, like turn them into real products and services.

Yeah, I think that phase transition is something that we're really interested in because I think that's kind of also the phase transition where someone becomes like a prime factory user in our perspective. And so we want to make sure we can also cater to them before that phase transition. But really seeing that transition happen is something I'm really excited about because we haven't deployed to nearly enough, you know, small companies. So really, really excited for that.

I was reading an article this morning, I believe it was Lovable. Basically, I think they brought it to a K-8 school and the idea was like, these kids are so much better at using Lovable than adults, right? They have no fear, imagination is kind of running crazy. And I think we had another conversation conversation, Logan, not too long ago about how like almost when things are so simple and easy, like you kind of have a roadblock of like, all right, what do I do? How do I use it? So I kind of going back to those drones too. Do you, I guess your thesis around different use cases and how you originally thought they'd be used, is it pretty on target? Are you seeing kind of use cases come out from some of these enterprise organizations where you're like, crap, I didn't even kind of think of that. And now again, it's kind of taking wind. Like here's to get your feedback on just use cases and what people are seeing from a value perspective. In general?

Yeah, yeah. I mean, I think one of the biggest surprises, so we talked about, you know, PMs and technical PMs using Factory a lot in the enterprise. That was something that we actually weren't really expecting, nor were we planning it. We kind of, we saw with this one organization that we were working with that had about 200 engineers. There's one day where like, I know I like obsessively just refresh all of our metrics just because I'm just OCD like that. And one day they're like daily active users shot up by like 30 and we had deployed to every engineering team. So I was very confused what was going on. I sent a message to the CTO there and I was like, hey, like, who are all these people? Do you guys just like acquire a company? Is like that who these people were? And it turned out that the PMs kind of caught wind of this and they all, you know, requested access. And I think what was surprising in the moment, but retrospectively makes a ton of sense, is we've actually found that PMs take to Factory even easier because they don't have an ingrained software development lifecycle. That they've been doing for 15 years. And so for the developers, we kind of have to earn the right. We have to show, hey, you know, you've had this workflow for the past 15 years. Here's why this is going to make you more productive. Like we need to meet them where they are a little bit, show them kind of ways to come in and like slowly start adopting Factory. Whereas the PMs, they sound like they use the IDE ever. It's not like they have this like kind of rigid path that they've carved through. Their development. And so for them, they'll jump in and like figure out the discoverability that they have for like features that are like niche is way higher because for them it's like, here's this whole new platform. Like, what does that do? What does that do? I'm going to try this. I'm going to try that. And it's been cool. Like almost every single, or especially in that size of like the, like 200-ish engineers, almost like clockwork, we've seen we never particularly reach out to the PMs. They end up catching wind, they end up using it. And then we get a message from like the VP of Eng or CTO being like, hey, by the way, our PMs are submitting PRs now and they don't need to annoy our, you know, full stack developers to like go and change this thing on the front end or that thing. So that's been an exciting thing that we've seen. And now, you know, as part of our org-wide deployments that we do, we'll typically ping like the, you know, head of PM and be like, hey, by the way, We've seen this work before. You guys might want to come try it.

I love that. As a PM, that sounds like a great success story. Do the droids have names?

Yes. So I mean, we'll, you know, generally it's like, it's the name is based on the task that it's for. So like the code droid, but they need cute names.

That's, that's the thing. I feel like not even cute names, but I do think like a non-trivial amount of success of like Devin as an example is just that it like has a real name and like people can just like associate this thing like a coworker or whatever it is? Yeah.

So funny enough, the source of the name Droid. And so originally when we incorporated, we were actually the San Francisco Droid Company. We were advised by our lawyers that Lucasfilm is a little eager to claim that copyright. And so we were like, okay, let's change our name. So we became the San Francisco AI Factory. But so many times, like towards the end of a POC or if someone's sharing like good piece of feedback, they'll be like, I speak for the droids or these are the droids we're looking for or something like that. Generally the developer persona, they really, they really take to take that name. So for a while we were like, yeah, we're going to have to change the name eventually. But I think, I think it's, it might be sticking.

So yeah, but I'm just saying like give them, give them like cool personalities. I feel like that would, I feel like that will honestly go like kind of like what has happened with like Claude and like how like the model personality piece, like people are associating with Claude because of the model personality. I think there's a bunch of efforts like that on the OpenAI models as well. It's been interesting that like, as the models become more partner, thought partners, or like execution partners, whatever it is, that personality piece matters a lot. And I feel like you have a lot of degrees of freedom, like controlling this UI surface to be able to sort of land that message to them. So.

I feel like it especially comes up for the review droid because a lot of people have fun being like, oh, I'm getting roasted by the review droid, or can it be nicer or can it be meaner? So that's, yeah, that's, that's definitely something that we're thinking about.

This is a good like content licensing. I'm thinking about like, like old GPSs, remember how they, you could like pay extra money to have like, I don't know if this was a real one, but like Snoop Dogg as like your GPS thing or something like that. I feel like this is like kind of a similar vibe where like the personas could be really, really interesting. And if you get like the— I feel like the right, like, funny technical personas, I feel like—

get like Jeff Dean as your, as your droid that's, uh, looking over your shoulder.

Yeah. Or the guy— the people from Silicon Valley, like the ones who always do those like fake videos. I forgot, uh, the—

some of those people.

That'd be, that'd be super interesting.

Yeah, I love it.

Um, I, I have a question really quick, and then Nolan, sorry, I'll, I'll throw to you. Um, model progress from a coding perspective, like how much has this— like, obviously that's been like the dominating narrative of the last 3 months is just how much progress there's been with all the vibe coding stuff, with everything that's happening with text-to-app and just generally models being good at coding. How has this— back to this thread of like you refreshing metrics all day— have you seen like usage of Factory because of this? And I assume you're on whatever the latest models are from whichever provider, just like going up and to the right because of how good the models have become.

Yes, but it's also, it's a little bit of a double-edged sword. So on one hand, it's great because as the models get better, we kind of get like free performance boost. But on the other hand, as developers change their behavior and become factory users, they also get really used to the given model that they're using. So a couple months ago, I think generally the standard was like SONIC 3.5. That's really been shaken up in the last few months, but everyone kind of got used to it and they understood how to interact with it to get generally like just a familiarity that you would have with like a human that you work with. Then recently with like Sonnet 3.7, with Gemini 2.5, with O3, it's thrown a little bit of a wrench because even the way that these models use tools or the way that they're— you can see how they're RL'd a little bit differently. It's an interesting question for us, which is how much of that change should we be like shock absorbers for and make it so that users of Factory don't need to adjust their behavior? And how much of that change do we actually transmit to them? Because maybe this is the direction that models are going and they should adjust the way that they're interacting with them. This is super not obvious. I think this is a— it's a little subtle, but when you have developers who really do change their behavior, like clockwork, like when we— if we change a default model, they'll be like, hey, like it feels like it's being different. Like, why is it not as eager to do this thing or that thing? And it's, uh, it's super, super interesting.

Um, do you let developers or users, uh, like pick the model? Have you just thought about like a sort of— I feel like it's so easy now from a product perspective, just like outsource this problem to their users, to your users, because like it's so many products have that now. Have you, have you thought about doing that?

Yeah, so we have a model selector so you can pick whichever. We do have a default, um, but that's been— and the default again for a while was 3.5. Everyone got used to it. They would try, uh, different models every so often. Generally, like, if they needed more context, they'd switch to Gemini. Sometimes if they were feeling fun, they'd switch to like O1. But generally, you know, that's, that's kind of what that looked like. Now it feels like there's not a consensus among our users of which model they want to use the most. And I think now part of that is because, so if we are providing the ability for users to select a model, it's an interesting question of should we make it so that the user can ask the same— can they prompt in the or, or converse in the same style regardless of model and expect the same output? Or should they need to kind of adjust the way that they interact based on the model and like force that knowledge on them? I think that's a little difficult. But there you kind of, to get the best out of O3 versus 3.7 versus Gemini, you kind of do have to interact slightly differently. And it's, I think that's something that's, that's, uh, pretty top of mind for us right now. We've been defaulting to just like, we'll kind of absorb all of that to make it as unified an experience for them as possible. Um, but also as there are more and more models and more and more versions, and some people want this one because it's cheaper or this one because it's better at reasoning, uh, it kind of, uh, the permutations explode a little bit.

And so, yeah, I don't know how much y'all do this type of like technical content, but I would, Personally, as a user and someone who likes reading interesting things about models, I would be super curious if you have 5 examples in practice of where user behavior really has to start to diverge for the different models. I intuitively 100% agree with you. Obviously, the models are trained differently, they're on different data. I would expect them to be different. But I even have a hard time articulating. Intuitively, you would also think they should generally be able to grok similar sorts of So that'd be a cool— if you're taking requests for good blog posts, I would love to.

Yeah. It especially gets complicated with like custom tools that you give it because that's when it gets even like you get even further on the boundaries. Yeah. Would love to jump into that.

Question 2, how do you think product velocity will change as soon as developers don't have to touch every line of code? Because I think for me, it's hard to comprehend. Like you have all these developers right now, they have like basically this magic sauce. Like, does that mean corporations are just pumping out different features and different products here and there? Like, how are you seeing that in practice? And like, kind of what is your, what is your vision on that moving forward too for both large and smaller companies?

Yeah, that's a great question. I think it raises dramatically what is table stakes for velocity in terms of shipping things, but it also puts so much more weight on good taste. Um, because now having good taste sometimes requires a lot of work. Now it doesn't. Now, if you have the best ideas and the best taste, you can implement it relatively quickly. Somewhat similar. I can't remember who mentioned this to me, but it was, I really loved the analogy, which was to like early websites, like way back when people would put in so much effort to make, like in the early 2000s, would put so much effort to make a really nice website. Now you can do the fanciest stuff on your website and it could still look horrible. And so at the end of the day, what are the websites that you remember? The ones that have great taste. Great understanding of like the user that's coming in and how to educate them on whatever journey it is that you want them to go through when they get to your website. And so I think it's somewhat analogous here with product, which is, you know, before there might be a product that took 1,000 developers 2 years to build. Now maybe 1 person could build it in 6 months. And so what is table stakes is now like, you know, the amount of engineering work you can get done. So it's kind of all the more important that whatever you do end up putting your resources on, you have that core sense of taste. And I think that's actually, it's really good for the world because there are probably so many people that have incredible taste that are kind of just like latent within society who maybe don't have the resources to build based on that taste. And with the way models are going and the tools that are coming out kind of lowers a lot of those barriers. And so then hopefully a lot of the software, and even eventually, you know, this will obviously come to hardware as well, uh, the software and hardware that we consume will have better taste. And I think that makes me happy. I think that makes a lot of people happy, even if they don't realize it.

Another quick throw-off, this, Logan, sorry, but I, I think, you know, from someone who's building, someone who is a founder, like, I'm curious to get your perspective on this too, because it sounds like obviously with this you're able to turn around things a lot quicker. I think there's an emphasis on that feedback loop for different products, like, hey, what are our customers saying? What is the community saying? How can we change it? But I think too, especially with so many companies and products coming out now, especially around AI, like, I do think there is like a fine balance between, hey, do we just push something out there, let the community give us feedback, and then try to go reiterate? Or again, for me too, like, I look at all these different newsletters or blogs, I'll try out a tool, I'll basically give like 5 minutes. If I really don't like it, I probably would never look at it again. There's so many different options. So how do you kind of strike that balance between Hey, you know, we can be quick and agile as far as trying to get something fixed, but at the same time, we kind of only have one opportunity, maybe two, to try to hook these people in for the long term.

Yeah, that's a really good question. I think that's kind of— I see that as almost part of the brand. Like as part of your brand, you can have either the, hey, we're quick and dirty when we ship something. There could be a little, you know, some stuff broken, but you know, you're getting stuff hot off the press. As soon as we kind of get, come out, like develop it, I guess. And then there's the brand of like, no, you know, we're kind of gonna really digest things before we put something out. And so I think that's kind of, on one hand, it's a branding choice. I think on the other hand, it's also a choice of the external constraints that you choose to put on yourself. And so this is part of why we chosen to be kind of enterprise-focused is because if you're— this space is obviously very new and it moves really fast. If the payment cycle that determines the churn, that is like the thing that's gnawing at the back of your mind, that's like, oh, I've— if I don't come out with this feature, they're going to switch to this other tool that does very similar things to me. You basically have a monthly cadence by which you can make bets and make product decisions, right? Because a lot of these tools, it's like, they're super neck and neck. And it's like, well, this person did this, and we're going to lose half our users if we don't do that in the next payment cycle. So we have to do that, right? It's kind of a race that we wanted to avoid. And so focusing on the enterprise, our kind of characteristic time period with which we can make bets goes from monthly to yearly. And I think that's especially in a space that moves this quickly, it avoids some pitfalls that might be trendy for one month, but not really relevant for a quarter or for 6 months or for a year. Yeah, but it's a difficult question. I think it's not an obvious answer.

That's a very interesting take. No, I think that makes sense.

I think that's spot on. I think this is one of the biggest challenges of the— and it's my general sentiment, and I haven't been a part of any of the other platform shifts, so I don't know. But it's just the race dynamics where it's like, as the race is very clearly still early, it's like you are incentivized to not let, you know, from a model perspective, product, whatever the thing is, like just not let your competitors have something that you don't. And it makes it hard. Like I, you know, I feel this pressure in my life of like, it makes it hard to have a long-term view of what a great product looks like or what the North Star looks like. So I'm happy that other people feel that pressure as well. But John, how do you think about like, and I'm curious actually, back to this thread of like product velocity. Have you seen, or, and maybe we're still too early for this, like, have you seen the pace of engineering change or like product development start to put pressure on like other functions inside of organizations where like the product team and the engineering team are moving so fast and it's like, you know, how do we even market this because it's changing so quickly and we're delivering on the roadmap? So have you seen that like start to play out at all?

Yeah, so I think not super strong signal to the extent where it kind of goes into things like product marketing and that sort of thing. But definitely in the relationship between engineering and product, for sure. The kind of the table stakes of what is like a PRD has completely changed. The way people think about like coming up with mocks or coming up with like proofs of concept. That has really, really dramatically changed. Kind of, you can't I think gone are the days of like, you know, a PM coming in with like an idea just in like some bullet points and some quotes from customers. It's like, go mock it up and show us and it better be interactive and we better be able to see and like start giving it to actual customers and let's see what they think. Like, I think the, the, there's just so much less latency between idea and getting validation. And eventually shipping it to production, that those cycles compress. I think we haven't seen it hit the other orgs yet because of the fact that there's still— I think that there's still such a backlog of like ideas within engineering and product. Like generally the story of engineering and product is we have this many ideas and this many that we can do. And so maybe we're like kind of increasing that a little bit, but there's still Even though now there are more things you can do, there's still that taste factor, which is another kind of like area of that funnel, which is, okay, just because we want to do it and can do it, does that mean we should do it? Um, and the answer is maybe not always yes. I think it also will take a while for, uh, especially large orgs to get used to a new shipping speed because it's a little uncomfortable sometimes to, to like, if you're kind of used to going at a certain pace, it's a little uncomfortable to suddenly like ramp up the accelerator.

Yeah, you get a lot less sleep, unfortunately. How— one question that's always interesting to hear folks' perspective on, especially as you're sort of so deep building with the models every day, if you could sort of wave a magic wand and all of a sudden the models had a new capability or they were better at like one specific thing, what would that be from your perspective to like unlock either new customers that you can't serve right now or make the existing process better for your current customers? Is it model quality at coding, speed, etc., etc.?

I'm curious what that would be. Yeah, I mean, I think speed, especially the time to first token, is just such a nice thing to have fast. Pure, like, even if the quality is the same, if you just get that first token faster, it just feels better. It feels like you can kind of move faster. So that's one thing. And think in terms of performance, longer context is always great, assuming that the context is actually used well. I think sometimes, I think this was less so now, but a while ago there was like a lot of hype on kind of just like maximizing context windows as much as possible. But the reality is, okay, great, you can have a lot of context, but are you using it well? So it's, Large context that's actually used well, I think is really important. But I do think that will eventually have diminishing returns because I think if you have good enough planning capabilities, you can then subdivide problems enough such that you might not need to have ridiculously large context windows. To future-proof this though, I'm sure we will have ridiculously large context windows, billions and billions of tokens, et cetera. What was it? I think Bill Gates once, or was it maybe it wasn't Bill Gates, but someone said something about like, why would we ever need more than like a megabyte of memory or something like that? I can't remember exactly what that was, but definitely don't wanna fall into that trap. But yeah, I think having longer context that is very consistent and using all of that context is kind of an easy win. In the short term.

Kind of off the same thread Logan was going on to, I'm curious to always, you know, for people who are building, you know, what were some of the bigger bottlenecks or challenges you faced when trying to get it off the ground? And then also off that too, what was like the oh shit moment where you're like, this actually is going to get some traction and we could potentially be successful here? Like kind of both sides of the coin.

Yeah. Um, biggest friction point by far was we're not an IDE and we've been very opinionated on that for a while. And if you're a startup and you're coming in and saying, hey, like, you know, the thing you've been doing for the last 20 years, don't do that, use this instead. It's kind of a big ask, especially for the larger enterprise. So that's kind of like a big hurdle that we've gotten really good at because the reality is you need to meet developers where they are and kind of give them inlets into the platform where they can get that win and then realize, oh wait, maybe I should go use this again for some of these other things. So importantly, we do play very nice with the IDEs. We can sync to your local files so you can hand off at any point that you'd like. So that was one initial friction point. I think another is actually going back. Originally, the droids were fully in the background. There was no platform where you would interact with them. It was instead, you create a ticket to solve some certain bug and it would just go and submit the PR. The big challenge was we deployed this to the enterprise and we found that sometimes it was right, but generally if it was like a less important task and for the important ones, it might get 90% of the way there, which is great. You know, we spent so much time making the agent decoding really good. But if you have a PR that's 90% of the way there, it's kind of a pain cuz then you just have to go and pull it back down and work on it yourself. And then it's like, why did I even do this in the first place? And so basically to, at like around the end of 2024, we realized like we need to adjust course because Even though the engineering leaders like this, the developers themselves do not love what we're doing. They want to have more control. So we made a big bet, kind of shifted our entire focus for like 3 months on building this platform from scratch, which we, this was not something that we had done in the entire like year and a half that we existed at that time. And then really in January of this year, there was basically, there was a company that we were working with that really brought to mind why we needed a platform. And we ended up, it was supposed to be a 30-day pilot, it ended up extending to like 90 days. And at the end of it, like, we kept extending. I was like, no, I promise we're building this new platform. It's going to make up for the issues that we had. And then on the last week, we had a check-in. And it was like, suddenly it finally, like, everything clicked. The, you know, the pieces of friction were gone. They had that ability to adjust between that collaboration and delegation. And I remember almost crying on this call because I was fully expecting, like, they were going to churn and it was going to be We maybe made a mistake doing that whole shift in focus. And basically they were like, yeah, look, even if I was the only developer using this tool at our company, I told our CTO that we should still procure because I would rather use this than any other tool I've used. And that was really kind of the jumping off point for— since then we've deployed with dozens of enterprises and it's been going really well. And I think sometimes we as humans, or at least me, maybe, I don't know if other people are like this, but are really bad at understanding how things like will compound over time, even from like a product perspective. And so a year and a half, just like trudging through, having some wins, but just not that like obsession from developers. And then it's not like something like— I actually remember this, this guy asked if we changed the model that we were using and we hadn't, there were no model releases around then. And that was like suddenly just the friction and the paper cuts disappear and it just feels like there's a step function change. And that was, I mean, that was the best feeling. Yeah.

That's an awesome story.

I love that.

Are the last two questions that we'd like to ask everyone who we chat with. First one, what does your personal tech stack look like?

Yeah, so I guess this kind of varies day by day. At Factory, we use all the models, we use all the tools. I think partly because it's kind of our job to be super aware of, uh, of what's going on. Um, so, and really also we have customers that have super diverse stacks, and so for testing purposes we also need to have Slack, Teams, RingCentral, Google Drive, Notion, OneDrive. Um, so kind of there it's just a— like, if you look at my toolbar, it's just like a huge mess of a ton of, a ton of icons. In, I guess personally, in terms of the tool stack that I use, when I'm not spending time on Factory, which is rare, I tend to try and not use technology period. So like paper books, that sort of thing. I think it's always nice to have a little bit of the contrast there. But yeah, I think that's an overview there. I wasn't sure if you wanted to go particular on a particular—

No, that's awesome. I think that's a great frame of reference and an awesome answer. The last one is, what is, you know, we're sitting here in 2025. What is one thing you hope happens in 2025? And what's something actually that you also hope doesn't happen in 2025? And you can take it in whatever direction you want.

Oh, man. Let's see. That is a really good question. Do you mind if I take a moment to think about that?

Take as long as you need. We— this is— I always, uh, I've cringed in the past at us asking this question, but we consistently get— not to set you up for a high bar, but we consistently get just like really interesting and like very varied answers. Um, so whatever direction is helpful for you, I'm, I'm curious.

I guess the first thing that comes to mind, and I think this is relevant in particular for what we do and a lot of the messaging that other companies in the space have put out. I really hope that in 2025, people understand that developers are going to be more important. And it becomes clear to them, like we kind of dispel this notion that developers are going to disappear and all their jobs are going away. But instead, it's like, no, definitely study computer science. Definitely learn how to code. Definitely learn how to use these tools. There will be more need for developers. We will have better products, better software that has better taste. And I just hope everyone else kind of realizes that in 2025. Something that I hope does not happen in 2025. I guess, um, yeah, maybe the first thing that comes to mind is I hope that, uh, the United States does not lose the lead in terms of, uh, AI, building an AI. I think something that has become clear over the last, uh, 2 years, let's say, is that obviously we're putting in a lot of money, but I think we're not necessarily diversifying our bets enough. Um, and we've seen like some labs that have much more constraint than US labs come up with innovation almost because of that constraint. And I think making sure that not only do we like put the capital necessary to invest in these AI labs, but also into the maybe less trendy bets. Actually, I know this viscerally from theoretical physics, which is what I did before. And in the US, like theoretical physics and in particular string theory, there was a tendency for— they're basically— there were two— there are two professors who are kind of like the heralded, like everyone wants to do what they're doing. And there's really, really bad, like herd mentality of like, whatever they're working on, everyone else wants to work on. And I think it's just important to diversify the bets a little bit. Because obviously, you know, what the two smartest people work on, they're doing it for a reason. But chances are, they're not thinking of everything. And I think diversifying the bets on AI are pretty important.

I love that. That's a great— that is a great place to end. Matan, it was wonderful to see you. It was great to catch up. Congrats on on the launch. Hopefully we'll have you on again a year from now and we'll talk for hours about all the success of everything that's happening. Yeah, thank you for spending the time with us.

Thank you so much for having me. Really, really great chatting with you guys.

Thanks.