OpenAI Codex Team: From Coding Autocomplete to Asynchronous Autonomous Agents

In my opinion, the easier it is to write software then the more software we can have right now if we think of like I bet you if we look pull up our phones, well, you folks are investors, but if you're not an investor, I bet you if you pull up your phone most of the apps on it are apps that are built by large teams for millions of users and there's very few apps that are built like just for us and the specific thing that we need. And so I think as it becomes more and more practical to build like bespoke software for people or teams, we'll end up having higher and higher demand software.

Welcome to Training Data. Today we're joined by Hansen Wang and Alexander Embarikos from OpenAI's Codex team for a fascinating look at the future of software development. Codex is OpenAI's series of AI coding tools that helps developers delegate tasks to cloud and local coding agents. Unlike the original OpenAI codecs which was developed in 2021 to auto complete lines of code, the latest evolution of codecs can complete entire tasks for you autonomously in the background. The key difference between O3 and codecs is that while O3 is great at competitive programming, codecs has been rl tuned to be great at day to day enterprise development tasks. Alexander and Hanson share more about the backstory for codecs and the broader paradigm shift from snappy autocomplete to longer running background agents. Plus they share their surprising vision for how developers will interact with AI in the future. As sync and async experiences merge, it might look more like TikTok than your current IDE.

Thank you guys for joining us. It's wonderful to have you here.

Hey, thanks for having us.

Great to be here.

We'd love to hear a little bit more about what you guys work on. Tell us about the Codex team and your story.

Well, yeah, I'm Hanson. I'm one of the researchers that helped train the Codex 1 model and I'm.

Alex, the product lead.

I think for me the name Codex is such a great callback to the original Codex model. That was kind of like an aha moment for me when it first came out because I think GPT3 was really cool. But then Codex was like the first moment where I felt like it's like, wow, this can really do something that is going to change the world. And that's actually kind of like how I got into the whole startup space. One of the first couple demos I did was using Codex to do data analysis. I think it's actually a funny Story I was here as part of Sequoia's ARC program. That's how I met Lauren. And then one of the demos we did, we actually used OpenAI codecs to do data analysis and that's how I started in the startup space. And I think as time went on, as the later versions of GPT came out, it became super clear that using AI for agentic use cases was going to be the future. And so I joined the company to work on agentic coding efforts.

Yeah, and this was like a per standard OpenAI style where we like the naming to be as easy to follow as possible. This is the Codex of like, I think it was 2021.

Yeah, this is pre chatgpt, right?

Exactly. Yeah. So it was actually like the model powering GitHub Copilot. And then recently as we were working on this product, which we'll talk about, we thought, you know, this is like a super fun brand. Also a very apt name, you know, code Codex, code execution. So like we decided to sort of like resuscitate the brand and like keep using it.

You said resuscitate. So was Codex dormant for a while and then you all resuscitated it for another?

Yeah, we haven't used the brand like recently.

Okay, okay. Really cool. Can you tell us a little bit about Codex the agent and what it does?

Yeah, I think so. Basically Codex is a coding agent that has its own container and its own terminal, kind of like fully in the cloud. You give it a task and it comes back to you with a PR in this sort of like one shot style. And we actually experimented with a lot of different form factors kind of along the way, but kind of in the end decided to settle on this one.

Yeah, so we've been working on a bunch of agents and we've been working on a bunch of coding products as well. And basically in our mind, Codex is like this thought experiment for how would it work to code with AI, but where we sort of put all our effort into thinking about what would that would feel like if the AI is working on its own computer independently from you, and so you're delegating to it rather than pairing with it. And so some of the things that we're really proud of with this Codex launch are thinking about the compute environment and how do we set it up so that the agent can actually work on its own, but be productive and creating the model which Hans can talk more about, basically that isn't just good at writing code that looks good or is functional, but also is really good at writing code that is useful for professional software engineers and mergeable, ideally without even touching their own computer.

So what is the difference between codecs and codec cli?

Yeah, we've definitely gotten some questions about that. I promise this is all going to make even more sense over time. So, basically, Codex for us is our brand for agentic coding. And we have this vision of we're going to have this agent, and mostly the agent will work on its own computer, but it should also be able to meet you in any of the tools that you use wherever you work, be that your terminal or your IDE or your issue management tool. So codec CLI is basically like Codex, like in your terminal. So, like, CLI stands for command line interface. Right. So it's like in your terminal you can work with Codex. That's like your environment. And then Codex or Codex in ChatGPT is basically a codex working on its own computer today. Those are just distinct things. As a brief aside, one of my favorite things about working at OpenAI is how willing we are to cut scope and just like, launch things quickly, but over time, we'll actually bring those things closer together. So you can really think of it as just like. It's just like Codex, and it can be in ChatGPT or it can be in your CLI.

Very cool. What did you have to do differently for the model to make it useful beyond just writing the next line of code?

Yeah. So I think one of the most interesting progressions. So if you go back to the 01, the first reasoning model that we launched, we highlighted how good it is at math and even coding competitions as of now. I used to be a competitive coder, and it's better than me at competitive coding. It's better than most, almost all people at OpenAI at that. But I think one of the things that we saw was that despite being good at these programming competitions, it wasn't actually that good at producing mergeable code. And so we even highlighted this in the blog post. With models like O3, the code that it generates often isn't quite to the taste or style that a professional software engineer would expect. So. So a lot of the effort that we spent on training this model was aligning the model to basically the taste or the preferences of professional software engineers. And that's something that took a lot of, I guess, specialized training.

Yeah, I have this very producty analogy that I like, which is like, if you take our Riesnik models, which are great at coding, they're great at coding, but it's kind of like this really precocious, competitive programmer, college grad who doesn't have many years of job experience being a professional software engineer on a team. And so a lot of the work we did to go from 03 to codecs one was actually the equivalent of those first few years of job experience where it's like, hey, what does a good PR description look like? PR titles. How do you read the style of the code base and then make sure your code is in the same style? How do you test well? How do you show that you tested well? Stuff like that.

What's typically the aha moment for when somebody uses codecs?

Yeah, I think one of the things we have in the onboarding is like find and fix a bug in the code base. I think that's one of the areas where codecs really shines is specifically bug fixing. Just because it can actually independently try, not just to see if something looks a bit off, but it can actually go and then verify that, okay, I can try and reproduce a particular issue. And so I think even leading up to the codec's launch, there were a couple of bugs where we were sitting there wondering what's going on. And honestly, sometimes the easiest thing to do is just paste in a description of the issue into Codex. And we were surprised how frequently that would actually end up with a usable fix.

Yeah, fun story here. Hopefully this doesn't give away too much, but at 1am the night before launch, or the morning of launch, at 1am we were looking at a bug with an animation, a lottie animation. And this is the kind of thing like, okay, I guess we could cut it from launch scope. It'd be okay to launch without it, but we really wanted to get it in and we just couldn't figure this out. And so an engineer ended up describing what the bug was and putting it into Codex. And actually, a fun pro tip for anyone who's using Codex is that if there's a really hard task, it can be useful to ask Codex to take multiple cracks at it. So they pasted that description in and ran it four times, like, hey, there's this bug. We can't figure out what's going on. And three of those rollouts did not work. And then one of the four was just like the fix of the bug that we were stuck on for hours at 1am before launch, and so landed. The fix deployed the code and the animation was in for launch.

That's awesome. Maybe tell us more about how you all are using it internally at OpenAI is every engineer is every researcher using codecs now in their workflows.

Yeah. And actually, can I give you the other kind of magic moment?

Oh, yeah, please do.

Definitely.

One of the interesting things about codecs is that it's a very different form factor from maybe what people are used to. A lot of the AI products that people are used to, especially in software, maybe GitHub Copilot was the first really good one. There are really things that kind of work with you in flow and you're just kind of seamlessly going back and forth. You're kind of pairing and it's flavor zone pairing. And we think that's awesome. And the codec CLI is a tool that you can use in that way. But for codecs, we really wanted to push this idea of you're delegating because in the future, we imagine that actually the vast majority of coding is actually going to be done independently from the human working on their computer, who can only do one thing at a time. And so basically it'll be done by agents working on their own computer. And so that is a very different thing to delegate to an agent than it is to pair with an AI model that's in your tooling. And so you have to kind of use it differently. And so when we actually were working on an Alpha before launch, we would just give this agent to people and be like, hey, just use this however you want. And we noticed that many, many of the people trying to use our Alpha of codecs were just not really finding it super useful. And then we're like, huh, that's interesting. Let's look at how people at OpenAI are using internal tooling like codecs. And we realized there was a big difference, which is the mindset of using it. The mindset that works really well for codecs is kind of this abundance mindset. And hey, let's try anything. Let's try anything, even multiple times and see what works. It saves me time. And so we've kind of shifted the way that we even onboard people into the product to try to create this aha moment, which is running many tasks in parallel. So, like, for us, if we see someone trying it out and they've run 20 tasks in a day or an hour, that's amazing. And they're probably going to like, they've understood basically how to use the tool.

Fascinating. How does that change the role of the human when you have to review all of this code? Like, if two of the three work, then what do you do?

Yeah, I think we've put A lot of focus on also making the outputs easy for people to review. So one of the things, things that we're proud of is we haven't seen this in too many other tools is the ability for the model to cite its own work. So not just the files that are changed, but also even the terminal outputs. So if it ran a test, and for some reason test wouldn't work, it actually tells you that, and it tells you, here's the exact terminal command I ran, here's the output, makes it much easier to verify the outputs. But it is a great point. I think we're shifting to a world where a lot of the time that we spend normally coding, a lot of that is going to shift to actually reviewing the code.

Do you need humans to review the code? Because I think of code as one of those things where it compiles or it doesn't, and once it compiles, you can go and check if it does the thing it was supposed to do. Do you even need humans to do the code review?

Yeah, I mean, for the foreseeable future at least, I do see that to be the case. I think a lot of it's also just like building trust with the early users. I think people really need to have a feeling for what things are working well, what things are not. And I think there's always just some external context about what makes this code correct that might be beyond what you initially provided as context.

Yeah, if you think of what a developer does, and this is obviously oversimplifying, but there's like, okay, there's coming up with what things maybe should be done, discussing them with the team, maybe deciding what to do, call that ideation, maybe. Then there's design, like, okay, what are we actually doing? And then planning, how are we going to do it? Then there's implementing and then validating, testing those changes. And that's basically a loop. And that small loop of implementing and then testing is what Codex is great at right now. Although we can talk about how you can use it for planning, too. And then there's actually deploying the code and then maybe maintaining the code, writing documentation, et cetera. And so I forget the exact stack, but I feel like a stat I remember recently is engineers spend maybe 35% of their time coding. It's not actually the majority of even what engineers do. And so the future that we're trying to build towards is one where if you're a software developer or even in any profession, all the work that is easily automatable, that's usually Grungier type of work you're not doing, you're delegating that. And then the work that is more interesting, because maybe it's ambiguous or maybe because it's really hard, that's the work that you're driving. So we're trying to build towards that work, that world, and I think we have to get there iteratively. So, for example, right now, if you're a human and you write code, another human is going to review that code. Right. And so we're not going to come in and just try to change that. We're like, okay, let's plug into that. So the way the product works right now is like, you, the developer, are being accelerated by the tool. You ask for some code to be written, you decide if it's good and you want to push it out to your team, and then your team can review it. And then over time, we'll basically kind of expand what we can do. So we'll help more and more with planning, maybe even designing, maybe even thinking about what to do in response to things that are happening in your app or at work. And then we'll push to make review easier and easier, as Hanson was describing.

Yeah. And I do think I see a future where you have multiple agents collaborating together. So you have the Codex agent writes the code, and then maybe the operator agent's the one that's testing it. And all the different agents that we've been working on at the company can kind of come together.

That's awesome. Have you seen people now that you can delegate doing, writing code? People beyond engineering teams start to use codecs, and we get into the world of vibe coding. You guys are helping us bring us further down that hole.

Yeah, this is actually super funny. We were. So the answer is yes. But I'll tell you a story. We were working on our launch blog post with Lindsey here, and we were talking about what quotes to to quote from customers. And we had a customer that wanted to say, yeah, we on the engineering team love this. And also, it's like a power tool for PMs. And I remember looking at that quote and being like, this is a really cool quote because I'm on the product team and I use it to just avoid having to bug an engineer about things or to answer questions. But I remember looking at that quote and being like, do we want that in the launch blog post? Because the target audience for what we're building is specifically professional software engineers, not vive coders. So I think we ended up not including that exact line But I think over time, as we have agents that can help us code, I would expect more and more people to be able to contribute to code bases. Yeah.

You think number of professional software developers goes up or down over time?

This is just my opinion, but I think it goes way up.

And not vibes. Coders, Professional software developers.

Yeah, I think so. But yeah, in my opinion, the easier it is to write software then the more software we can have right now if we think of like, I bet you if we look, pull up our phones, well, you folks are investors, but if you're not an investor, I bet you if you pull up your phone, most of the apps on it are apps that are built by large teams for millions of users. And there's very few apps that are built like just for us and the specific thing that we need. And so I think as it becomes more and more practical to build like bespoke software for people or teams, we'll end up having higher and higher demand for software.

Yeah, as I think about how I use it, I think it just really is a multiplicative factor right now rather than any sort of replacement, especially looking at the patterns of our internal power users. There's a really dramatic difference in the top users of codex are doing 10 plus PRs every day. It's just really such a multiplicative factor that I can't see a world in which it's lowering the bar to creating software so much.

That said, I think this is a really important question and to be completely honest, we don't know. And so this is something that we as a company pay a lot of attention to.

I want to talk a little bit about what's happening under the hood on the technology side. So you mentioned that the model itself, one of the things that makes it different from competitive programming is you've made it more be good at the things that a professional software developer would do. Is that the biggest difference on the model side or should we think about it as a close cousin of O3?

Yeah, it's definitely the same model as O3, with additional reinforcement, fine tuning. But that said, yeah, I think so. Part of it is kind of like these more qualitative aspects of what makes a good software engineer versus simply a good, let's say, coder, like style, even how it writes comments. I think that's one of the things that people have noticed with other models. And then on top of that, I also want to highlight one of the big challenges was making good environments for the agent to kind of learn in. And so if you think about real world software repositories, it's so varied and complicated. Think about how much DevOps has to go into setting up a repository. And that's something we're kind of learning the hard way with our environment setups.

Should we talk about the multi repo I was showing you yesterday? Oh yeah, I was showing Hanson the repo for the startup that OpenAI acquired. And so we joined and so we were looking at that repo together, thinking about it for use as an environment. And Hanson's like, so where are the unit tests? Because the agent uses unit tests to verify. And I was like, this is a real startup that has no unit tests.

I mean, almost the same, so I can't complain. So yeah, you have all these really messy environments. So we had to, over the course of training, we had to basically generate these really realistic environments for the agent to learn from. And I think one of the reasons that we're able to make such an end to end product work is that we have the same environments that we use during training and the same basically this containerization infrastructure that we're using to serve in production. So our users are, we're running our own compute environments. When users use codecs, they're running in the exact same environments that we're using to train training.

So you don't have the agent saying, but it works on my machine.

Exactly.

Okay, good. I think these are also the longest running agents I've seen out of OpenAI. Deep research maybe was the previous one that was longest running. And my understanding is codecs can sometimes spend 30 minutes on different tasks. Are there any kind of surprising challenges and things you've encountered? Just getting inference time to scale up on a query for so long?

Maybe I'll start with the product side and then there's many on the modeling side, but on the product side, actually the thing that I think the most about is user intent. It's like actually if you imagine someone using autocomplete in their ide, it's not super hard necessarily. I mean, obviously it's difficult, but it's not super hard to predict what are they trying to do right now for the next microsecond. But for doing a task that takes 30 minutes, it's actually fairly difficult to help a user describe the task. They may not even know exactly what they want for 30 minutes worth of work. Something that we spent a while debating, and it's still a thing we debate, is what is the right granularity of a task for someone to give to Codex. And how can we make it easy so that Codex can be really flexible where you can use it for one line changes, you can use it for big refactors that you know exactly what you want or like larger features where you know what you want. Or maybe can you use Codex when you don't know exactly what you want? And so maybe you should ask Codex for a plan and then you can have it Codex suggest tasks and then do those tasks afterwards. So that's still a topic of debate and iteration for us.

Yeah, I think that's actually like a good pro tip for using it. It's actually really good at coming up with its own plans. And then sometimes it's really tedious to specify everything you up you want upfront. And that's kind of like one of the unique challenges about working. Like if you want it to work for an hour at a time, then you kind of do have to specify a lot upfront. Which means that you have to spend like, I don't know, like 10, 20 minutes coming up with that. But if you use actually like the ask mode to first like, you know, generate like a high level plan of what you want to do and then you can like iterate on that with the model before you send it off for an hour.

It really is like working with an internal. Yeah. What about on the model side? Anything that's surprising in terms of model behavior as it starts to run for so long?

Yeah, I think our models have gotten a lot better at kind of like sticking kind of like on task, especially with these longer rollouts. I will say there are cases where there is a limit to the model's patience even though it's quite high. So it can be frustrating. Sometimes it goes off for 30 minutes and then you know that this is a case that we're wearing. Working to get better at where it's like, you know, it's kind of like just like a human. It comes back to you. It's like, sorry, I don't. This is too much. I don't have enough time to do this actually. Like that's one of the things it says.

Just like an intern.

So very, very human. Like in many ways.

Yeah. I'm curious how you think about the right interaction patterns and how they evolve and how the suite of products around this evolve over time. We have Codex, we have Codex Cli. What else do you think is out there in the design space for engineering and building products?

Yeah, so the Codex as we launched it is really just like it's a research preview. It's a thought experiment, a useful one, but it's still very early. And what we're most proud of with Codex is the model and the beginning of this foundation for compute environments. And the UI we shipped is one that we iterated towards. And there's some fun stories there, but. But it's definitely not the final form factor. And for those listening, basically, the UI we shipped is an interface in ChatGPT where you can submit a task and ask Codex to either answer your question or write code. And then you kind of have this something that looks a little bit like a to do list of things that you can go look at merging really, I think for. So we built that to really lean hard into this idea of an asynchronous agent that you delegate to. But what we want to build towards is a setup where you don't have to think about whether you're delegating or whether you're pairing with an agent. And it should just feel like working with a teammate, where that teammate is ubiquitously present in all the tools you work with. So you should be able to pull up any tool that you're working in, be it your terminal, your ide, your issue management tool, maybe your alerting tool, your errors, the tool shows you errors and just ask for help. Maybe even Codex has already taken a look before you even got there, and it has an opinion there. And you could be able to ask something, be it a short question or a long question. It'll just appropriately decide how much time to spend before answering you and just help you land those changes. So basically we want to kind of blend this idea of pairing and delegation. But the first thing we shipped was just the. The purest thought experiment. The other thing I'll add to this is one of the unique things about working at OpenAI is that we are the makers of ChatGPT, which is sort of the AI system that most people use. And so we don't actually see a future where as you go about your day, you're deciding whether to use the Codex agent or, I don't know, your shopping agent or taxi ordering agent. By the way, I'm just naming random things here, or you're marketing agent. Actually, the way we think this should work is you should just have one assistant that you talk to and you can ask it anything about anything and it can just do the things you need. And so that's chatgpt, that will become our assistant. And then if you're a power user of a certain type of tool, so let's say You're a software developer, you spend a lot of time in certain functional tools. Then you can go into that tool and have a bespoke interface with buttons, with lists that you can use to efficiently go about your day.

Do you think we'll still use IDEs?

Yeah, for sure, but they'll evolve. Right now they're very focused on writing code and as Hanson was saying, probably agents will be writing more and more code and so it's going to become there'll be a shift in emphasis towards landing code or reviewing code or validating them, or maybe even a shift in emphasis towards planning bigger arcs.

Yeah, I think we're already seeing a lot of people on the team. They kind of, first thing in the morning, they come in, they make coffee and then they kick off a few tasks just to kind of get a starting point. And then they come back after their breakfast and they look at the tasks or the PRs that got generated, then they'll take those and the IDE is kind of like the place where you take, maybe it will get you like 80% of the way there, hopefully, or even more. But then there's always this last mile where you go in and really fine tune based on kind of like your own vibes.

How do you see the broader market evolving? Like within OpenAI, you have so many different strategies here. And as you think about async tasks, as you think about some of the things that you mentioned moving into ChatGPT, we're seeing a lot an explosion of other tools and specialized models. You obviously are biased, but I'm curious what your read is of the broader market.

Yeah, it's a crazy time to be a developer right now. Like there are just so many new tools that are just so helpful. Like a fun story recently is I was in the airplane and there was no WI fi and I had thought that I was going to maybe write some code and like build a thing. And there was no WI fi and I was like, you know what, screw it. Like, it's just not worth my time to like even try to write code anymore. Whereas, you know, the startup that I was working on like many years ago, like part of the genesis of that startup was like me writing some code without wi fi in an airplane and I just wouldn't even do that anymore because like the market is just like, it's just changed so much. And I think this, I think we're going to see like an equivalent shift in an equivalent amount of time. So like in the next two years coding will look completely different. I think right now most of the tools that people spend, you know, that people find the most value from, are tools that work really closely with you, like in your development environment, you know, like basically pairing. And I think the shift that we're going to see, but we have to figure out how this will happen. But the shift that we're going to see is that actually the majority of code will be written by agents and those agents won't be working in your environment where you can do one thing at a time, but they'll be working in their own environments. And they won't just be triggered by you thinking of specific tasks, but they'll be connected into the tools you use doing work there. I think we'll see basically that shift towards agents. I think we're going to have to figure out a lot about code review as you were asking about. Personally, I don't exactly know how that's going to work, but I do know that Even already at OpenAI we're seeing much more code is merged by agents, but actually also even more code is generated by agents as folks are say, kicking off tasks four times to choose their favorite implementation. And so it's not 100% clear how we should even manage all this code that is being written. Some things that I will say though, in case it's useful to the audience, is that there are definitely things you can do to your code base to make it more addressable for agents. This isn't necessarily particularly novel, but obviously using typed languages is really helpful. Another thing that's very helpful is having smaller modules that are better tested. We joke about having good tests at all. Yeah, having tests. We joke about my startups repo, but I bet you we would have written it differently if we were writing it today. Even there's small things. The code name for this project is Wam. This is the coding for Codex. It's like Wham. And when we named it we were very intentional in doing so because we knew we would have code in the server for the website in various other places and we wanted it to be really easy for the agent to search for Wham related code and find it. And so we named the project Wham and we grep the code base first to figure out how often it was there. If we would have called it something like Code or Codex or Agent, you can imagine it would have been really hard for the agent to now you.

Called it Codex and now the agent's going to be confused.

Well, so in the code this is kind of my point, right. Like Intentional design in the code. We use the term wham like a lot because that's actually much easier for the agent to find. Obviously, if we didn't use a word like that, the agent could still find its way, but it would have to spend much more time to find the right files.

Yeah, it is cool that a lot of the things that actually make the code base easier for humans too also tends to make it easier for the agents. Like good tests, for example. Writing good docs is another great example where now I think there's even more of an incentive to do that because not only does it make your life easier, it makes the agent's life easier.

Okay, sorry to be the annoying vc, but Claude, Code and Jules are also like, I think, agentic coding experiences from others. Do you think the. I'm curious how you think your experiences compare today. And then do you think the market is probably going to converge towards the same vision of what sync and Async coding look like? And in that version of the future, what do you think OpenAI wins on?

I think we're going to see a little bit of everything. Even in what you mentioned, there's tools that are working on your computer, there's tools that are working on their own computer, as I mentioned, I think we're going to see the majority of work being written where the agent has its own computer. But it will still be really important for us to invest in accelerating developers who are doing work on their own computer too. Ideally, we get the best of both worlds there, but most work is done in agent compute.

I think the way I see it as well is I think one of the, the hardest part of software engineering really is taking all the context from the world and encoding it in these requirements, these design docs. And then the implementation, I think, as we alluded to earlier, is not actually that much of the lifecycle spent on physical coding. And so I think where ChatGPT shines is it is this assistant that has memories now. It has access to a lot of different connectors to all the different tools you use. We have operator deep research that have all these different capabilities. And so I think the vision where that all comes together is where a tool like Codex can really shine. Once it has access to all that knowledge, it's able to make use of that. And I think with that it should be able to do a much more effective job at just the coding part.

Yeah, imagine hiring a software engineer and the only thing that that software engineer can do is take a task from you and produce a PR or it has these very well defined features and it can exactly do those things. And then you ask for a random thing like, oh, hey, the team is getting together. Do you mind also, I don't know, getting a meeting room and leading a brainstorming. It would just be so frustrating if you hired a teammate and they refused to do that kind of work. Right. And so similarly, I think it's really, we're building towards a future where agents that you're working with are a little bit more generalized to reference. Hanson was talking about Operator and Deep Research. If you think Operator has a web browser, Deep Research has a different flavor of a web browser. Codex has a terminal. Really, your teammate has pretty similar tools like a human teammate. And so the goal for us eventually is to pick places where we want to really invest in a specific audience to make rapid progress. So obviously we're doing that with coding with Codex or GPT 4.1, where we generated specific evals for that audience and then made a better model for them for developers. But then over time generalize these things into simple things that everyone can use. So I think again with us with OpenAI and ChatGPT, I feel like that's a place where the products we build will look very different from something that's very only specifically for coding.

What do you think will be the primary UI that developers use to interact with codecs? Do you think it'll be chatgpt, the cli, the ide, all the above?

Yeah, I think a mix of all the above. I think we just kind of want to meet developers where they are in that moment. So it might not even be in the editor or in the terminal. It might be on Slack. Someone messages you, hey, there's a bug. And you're just like, hey, go fix it.

I'll give you my fun future UI that is not at all serious, but maybe the future of working with agents. If you're a startup founder in the future and you have a team of just you and a couple of co founders and many agents actually looks like TikTok, maybe you have Vertical Feed. And it's basically an agent has produced video that you can watch with an idea like, hey, a customer wrote in with this request, I think we should fix it. And then you swipe right to say, yeah, let's fix this, let's do this. You swipe left to say, no, we shouldn't, sorry, it's a hybrid. I didn't say this was going to make a lot of sense. And then you press and hold to provide feedback so you'd be like, yes, do it, but make sure the font is in italic. And so basically you have all these agents who are subscribed to information at your company or on your team, and they're proactively coming up with ideas and doing them and then giving you updates and you're kind of just like curating the work that is being done and.

They show you little previews of what the world could look like.

Yeah, obviously that's a half joke, though. I think that'll be kind of the arm's length working with agents and then it's definitely going to be really important for people to be able to go do the work themselves and pair with agents.

I get that it's a half joke, but it's a really cool visual because I think everyone agrees conceptually with this idea of collaborating and reviewing all the different changes an agent makes is going to look very different from how we code today. But nobody's actually given me a visual of what that might look like. So that's a really cool idea.

I love it.

Awesome. Should we wrap up with a lightning round?

Let's do it.

Okay. Recommended piece of content or reading for AI fans.

For me, that's like immediate. It's like the Culture by Ian Banks. Have you read it?

Yes, it's amazing.

Yeah, it is. A science fiction series started being written in the 80s and it is unusually positive in its view of how a future space faring human and non human race could kind of look. And there's a lot of questioning about what is the. The purpose and meaning of life when we have AGI.

Yeah, I think for me it's like Anything by Richard Sutton. I think that was my introduction to reinforcement learning. And I think it's kind of a joke here that we read the Bitter Lesson every single day. That's kind of the philosophy of OpenAI. I think even with Codex, we give it a terminal and it literally uses POSIX tools. That's the most Bitter Lesson way of working with the computer.

And your favorite AI apps gotta be ChatGPT.

Yeah, not ChatGPT. Come on.

They're so boring.

Okay, either it could be like a new feature that you guys have released other than Codex or something outside of OpenAI.

Okay. So I guess it's funny. I don't really think of AI apps, right. But I do like it when my life gets easier. So some things that I like are like when you're using AI, but it's kind of invisible. So like just I'm in product, so I often, like, file bugs and like linear has a really elegant integration where when you file a bug from a Slack conversation, it just generates the bug from the Slack conversation. But they never say AI anywhere. Just like you actually kind of don't even notice that it's using AI. Oh, wait. I came up with an answer for favorite AI app, Waymo.

Ah, there we go.

Yeah, I think. I mean, I think for me, like, Copilot has definitely been the thing that keeps delivering value every single day for me.

Okay. Robotics. Bullish. Bearish.

Bullish. Yeah.

Which new application or application category do you think will break out in 2025, other than coding?

Yeah, I mean, I think when you had Issa and Josh on, it's kind of the same answer, but 2025 is definitely the year of agents. I think we're going to see agents take off in a lot of different categories.

Yeah, have to agree with that.

What type of agents are you most.

Excited about, aside from coding Agents? That's a good question.

Well, so my take would be like, you know, if we. I know this is meant to be rapid fire, right. But, like, kind of the way we think of agents is you have reasoning models, right? And then you give those reasoning models, like, access to tools of the trade, and then you figure out how to train that agent to like, do the this sort of specific function. Right. So it's not just about writing, it's about journalism, or it's not just about coding, it's about software engineering. So that's kind of what we're doing. And in my mind, the reason I'm so excited about agents this year is because we now have a few agents shipped from OpenAI and other companies are shipping agents too. And so we're starting to see what kind of the shape of this is and starting to identify the primitives. And so specifically, what I've been excited about is as we bring this together and you come up with an agent that you don't have to provision separately for every single function, but it's an agent with a computer that has a browser and has a terminal and it can do multiple things without you having to exactly specify you are my coding agent or something.

Really cool. Thank you so much for joining us. Congratulations on what you've built at Codex and thank you for giving us a preview of how you think the coding market will evolve and also giving us a peek into how long running async agentic experiences will play out. Really appreciate it.

Thank you.

Thanks so much. Thanks for having us.

Thank you.