CEO Spotlight, Jay Hack, Codegen.com

Well, today I'm with my friend Jay Hack. He's the CEO and founder of CodeGen. CodeGen is an early-stage San Francisco-based AI-assisted code development platform that allows companies to refactor enterprise codebases at scale. They recently raised $16 million in seed funding. And prior to CodeGen, Jay founded and successfully sold his previous AI startup called Mira AI. And Jay is an AI researcher with expertise in natural language processing and computer vision. Jay, it's good to have you today.

Thanks so much for having me.

So first of all, let's describe the space as a whole and where CodeGen fits into it.

For sure. Code generation is a fascinating space and it's gone through, you know, a huge evolution over the last couple years. You know, kind of on the, on the, if you look at it in terms of layers, you've got sort of the foundation model layer. There's folks like obviously OpenAI and Anthropic, but also Poolside and other large labs that are specifically focused basically on making the best possible model. That can do code modifications, can understand code, this type of a thing. And they're probably less focused specifically on productizing that for developers to use in their day-to-day workflow. It's much more so the sort of like abstract model that you end up, you know, asking questions to, et cetera. And then, you know, potentially that will be plugged into a product workflow. And then sort of going up the stack, there's several folks who focus on in-editor things. So for example, Cursor, which I'm a big fan of, I use that every day. GitHub Copilot, which is probably the best demonstration to date of actually driving revenue from a code generation product. Codeium as well. The idea is you're a developer, you're in Flow, you have, you know, something that will do smart autocompletions for you or chat your code in the editor. But the focus is really sort of on this thing that's peeking right over your shoulder and helping you as you're iterating on code. And then on the sort of far end of the spectrum, if you look at the sort of, uh, attempts at fully automating the process of writing software, there are folks like Cognition, AutoGPT, etc., who are looking at basically having an agent that is fully automated and you can give it a code generation task like build me a settings page and it will go off and it will actually write code for you. And then I would say the sort of last category, which doesn't really fit neatly in the spectrum, is there are folks who are going after specifically finding a workflow that people do today and intelligently injecting AI in some way in order to speed up that workflow. And so examples of that might be things like, for example, Resolve.AI, which is actually right here next to us in our office that goes after the Incident management. So, you know, there's interesting ways in which you can have an AI go through and search through your logs and retrieve insights for you. And then us, where we focus specifically on the subset of tasks that are refactors. Refactors are those tasks that are very large in blast radius. And furthermore, there's no mystery in what the output state looks like. And so it's very easy for you to verify. And there's a lot of things you can do in order to inject smart static analysis, but also AI in order to basically get people to the goal line faster.

What would you say is hype? What's fiction, what actually works, what's useful, what's practical?

Sure. I think it's really important to talk about these things and be specific about timelines. And so what might be hype today is potentially very real in the future. And, you know, despite the fact there are barriers to working, we've certainly seen things that don't seem like they're going to work suddenly work 3 years later several times in the AI community. I think that, you know, the idea of a full-on agentic software engineer is something that really captures the public imagination. It seems like it would be true for people who don't understand code and who aren't senior engineers on large code bases. They love to imagine that, you know, these, oh, the things that these guys are doing will be perfectly automatable. And I think probably the most successful AI demo on Twitter ever was the full-on, you know, agentic software engineer thing from AutoGPT. I know Devin had one as well where you just sort of give it a task and it goes off and does stuff for you. And, you know, the truth is that there's, if you look at large enterprises especially, which is most of the code that matters in the world, you know, it's like the servers that are running it, Coca-Cola or something along those lines, there are very stringent requirements on what that code looks like, not just in terms of the style that it adheres to, but also the business logic. And so if you were to basically have, you know, an agent go off and make you a settings page, let's say for something like, you know, Amazon or something along those lines, there are so many business decisions that are required in order to correctly spec that out that it means that you can't just give a shortcut command to an AI and expect it to build something that's actually going to get merged into prod. There's what we call the specification problem, which is you need to be very specific about the things that it ends up doing. And a lot of the job there is just doing that disambiguation. And that is something that basically you cannot expect the language model to do in isolation. And there's a ton of touchpoints that are required in order to get it over the line to produce something that's actually valuable. And so for that reason, you don't actually see it being deployed in prod in real large organizations today. It's very much so the type of thing that basically lives in demo land and works for people who are building the zero to one for their product. And, you know, God bless, it's amazing. Like the fact that you can basically in this iterative loop build a front end very quickly, but we have yet to see that actually touch the enterprise.

Where do you see the actual value is in using AI and developer tools?

You know, I think that that is the value in using AI is the value in using any other technology, which is that allows you to be more productive per unit time. And so I think if you take that framing and it's like, okay, what things do we do today that are menial labor and that might be able to be automated? So, you know, for example, making small front-end modifications or speccing out front-end components. I think that's a beautiful use case. I think incident management. So like looking through logs and, you know, finding the little needle in the haystack that describes something going wrong. That's a perfect example of menial labor. You kind of just enter into a robot mind mode when you're doing it yourself. And then obviously refactors, and this is a shameless plug for our product, but we see it, you know, huge amount of people will basically go through and they'll have a bunch of contractors come in and they'll manually go through and edit file by file, the sort of rote program of what they're hoping to transform their codebase into. And that specifically is a fantastic application domain for AI because a lot of the time you actually, you know, basically know exactly what the output looks like. And so it's very easy for you to verify the fact that a state has been reached properly. In fact, you can even just run your tests on the output. But the actual process of doing that transformation is something that is very expensive. So that's the mentality we take towards it is find the things that are expensive today, however, are beneath the high watermark of the capabilities of AI, outsource that to AI and just make that part of an accelerated workflow towards getting toward, you know, a valuable output.

If you were to describe the state of the world before CodeGen, what does that look like?

I think you have a lot of people who are saddled with immense tech debt, and it's the type of thing they don't even consider fixing because the cost to doing it would be absolutely immense. And I kind of think of, you know, the game of modifying your codebase as a chess game where there are certain moves that you can make and each of those moves has a certain expense associated with it. And there are certain moves, like for example, upgrading yourself from an old version of Next.js to a new one, they're just so expensive it doesn't even come up in the conversation of, you know, what you're doing for your roadmap for the next quarter. Or if it does come up in the conversation, it's this massive political, uh, you know, situation you have to deal with, with your— or you got to get a buy-in from these people, get buy-in from those people, get resourcing for it, etc. Because there's just an immense amount of labor and work that needs to be done in order to effect that change. And we envision a world in which at least this subset of tasks where you understand that, you know, the output state is well-defined, the actual transformations that are necessary to get there are largely deterministic, but it's just very expensive and cumbersome to get your codebase into that shape. That is something that should be a chess move that is very cheap for you to do. You can merge it in an afternoon. You have all the sort of utilities there that are necessary in order to verify the change was performed correctly. Et cetera.

So, it sounds like the pain point, and I wanna ask you about that, is, can you fix the code, or do you just start over?

You know, I think that the prospect of just starting over is something that would not land very well with most of our customers. You know, we work with a 60-million-line Python codebase, and apparently, occasionally, in the conversations at this company, they say, "Should we just rewrite everything in Rust?" And I think, you know, that's just not viable. Like, if you have hundreds of millions of dollars of revenue flowing through a stack, you can't just rewrite it in Rust. You probably go after, you know, subcomponents of it and transform them over. But usually, you know, it's like you got to go through instead of around. You got to somehow find a way to make do with what you have and level that up. So that's really the type of problem that we focus on. There's something called second system syndrome in engineering where people always think the grass is greener on the other side, and wouldn't it be great if we kind of started from scratch? And the reality is that, you know, that's actually usually a much more expensive thing for you to do. And whatever whatever you have today is something that is working for you at the very least. And so you should level that up. Cool.

So how does CodeGen address that pain?

We have taken the observation that especially for these large blast radius tasks that you go after, the modification itself is usually largely deterministic. And so you can think of, for example, upgrading a package from version A to version B, something like moving your tests in such a way that they can be more efficiently run on different test runners because they're grouped in such a way they have similar dependencies. A lot of problems like this, if you look at the actual work that needs to be done, even though it is complicated, the transformation itself is largely deterministic. And when I say deterministic, I mean if you look at a codebase as a graph data structure, which it is, you've got, you know, files, they have imports between them, there's some control flow, there's a call graph, et cetera. You could write a program that is traditional, fully deterministic, that goes about modifying that data structure to be in a state that looks like what you want it to actually be in. And then it's very easy to verify that. And because of this property that a lot of tasks like this are, you know, can be broken down into something that's basically code moddable, if you will, The real alpha here is basically in going through and writing the thing that does the advanced static analysis necessary to figure out how to actually do those changes very quickly. And so we've invested an immense amount of time. We've built a compiler team internally that basically does that static analysis and understands these are the properties of this codebase, as well as wraps it in a very neat, basically API that allows you to express high-level and very powerful transformations and visualizations in a high-level Python SDK. So you can come in and you can, you know, say something very simple like delete all of my dead code, and this will be expressed in a 3-liner in Python, run that, and it will produce a pull request that is deterministic and furthermore guaranteed to actually produce a codebase that passes lint or is corrected, it compiles, et cetera. When you have a tool like this that basically allows you to express very complex transformations in such a way that all the edge cases are covered and the actual syntax for expressing that change is very short, that ends up being an ideal target for the output of AI. And so our customers will come in and they'll say, I have this complex transformation I want to do on my code base. They'll express that to an AI. AI will write a little short Python script. They can independently verify that script is doing what they want. They can run it, and then they can manually modify the code that actually produces that transformation themselves. So this allows us to basically make very large changes without having to make hundreds of thousands of queries to a large language model and deal with all the hallucinations and sort of unreliability associated with an AI. Instead, being just inject the agent into very rapidly coming up with a script that forms the output that people are, are hoping to achieve.

How do you prevent AI from hallucinating?

I think that you're never going to be able to 100% prevent the current generation of models from hallucinating or just kind of going off the rails. But what you can do is you can constrain the action space. Basically, the idea is you have the AI take actions in a way such that things that are wrong are syntactically invalid. So you can imagine playing chess with an AI, and the way in which it, you know, takes a chess move is it says A7 would correspond to, you know, moving a piece to A7 or something along those lines. And if it says X7, which is an invalid chess move, you say, hey, you can't do that, that's an invalid chess move. And you tell the AI that it has to take another move. We take the similar approach with codebases, which is instead of having the AI directly operate on the codebase and output a string, that might be a new React component or something along those lines, because straight up, it's only acting through our SDKs and our SDK, which, you know, understands the code as a graph data structure, you know, very traditional concept. It will prevent it from taking actions on the codebase that will produce an invalid codebase at the end. And so our SDK will yell at it and say, you can't do that, try it this way. And that basically allows the AI to iterate by itself towards a much more valid set of transformations on the codebase.

Have you figured out how much time is saved when companies use CodeGen?

So there are some cases where you can definitively point to the fact that there was a bunch of contractors who were being hired for this. They were spending 200 hours, which is projected, in order to do this refactor. And, you know, we did it in 15 minutes. That's great. And there's an actual number you can point to. There's a whole other class of problems, though, getting back to this sort of chess analogy, where people don't even think about doing it because it's impossible. Otherwise, there's just no solution that would allow them to do it. And so now we do it and there's a ton of value produced by it, but it's not as though you can point to a number of hours hours that were otherwise spent because it was just fundamentally impossible to do in the first place. And so in a way, I think this kind of gets back to the more general, like, debate about AI, where it's like, is this actually taking human jobs versus adding some fundamental new capacity, uh, you know, to what engineers can achieve in a day? And I think, uh, we sit kind of more so in the latter, to be honest. Um, you know, you just wouldn't imagine that you could accomplish these things without a tool like CodeGen.

So it just opens up the possibilities. You would totally think in a different way once you have this tool.

Exactly.

So is there an ideal prospect profile? How would you describe that?

For us, that ICP is the platform team at very large companies. So these are the folks who are in charge of maintaining good conventions around the codebase, managing CI/CD pipelines, making sure there's good developer experience, making sure there's good internal APIs that people can use in order to build good features for their product. And as a result, they end up having basically modifications they perform that are very large blast radius and will affect many, many teams. Those folks have been systematically underserved. You know, some open source ones that are certainly useful, but they definitely do not do— they do not have the full feature set that is required in order to make somebody in the ICP category that I just mentioned truly lethal. And so that's a great person for us to go after initially. We find that typically we go into a company and we kind of start with the platform team. They end up making a bunch of changes with our tool. Then our output will end up showing in the git blame, which is basically if you're in your editor, you can see who wrote what code and they'll be like, oh, CodeGen, what's that? And they kind of find a product for that. And then it, yeah. And then it snowballs from there. And then we, you know, sort of see people coming in organically.

This sounds super exciting and I'm excited to see the snowball effect take place as people use the product and love it and refer it and all that stuff. So if people wanted to try it out, where should they go?

CodeGen.com. Reach out. We'd love to have you on the platform.

Awesome. Awesome. Well, this has been exciting, Jay. Thanks for spending the time and we'll be in touch soon.

Awesome. Thank you so much. Appreciate it. Cool.