OpenAI Just Released ChatGPT Agent, Its Most Powerful Agent Yet

I think this model is actually very good at multi turn conversations and it's very nice to continue working on a task with. I think that's one of the deficiencies of deep research. A lot of people will do multiple deep research requests in a single conversation, but it doesn't always work so well. So I think we're really happy with this model's multi turnability and we just want to improve even further. And then I also think personalization and memory for agents will also be very important. And right now every agent task is initiated by the user, but in future it should also be doing things for you without you having to even ask in the first place.

Today we're exploring the evolution of AI agents with Isa Fulford, Casey Chu and Edward sun, the OpenAI team behind the new ChatGPT agent. You'll learn how they got to a huge leap forward in capability by unifying the architecture across Deep Research and Operator. Allowing for multiple tools to share state, giving users fluid transitions between visual browsing, text analysis and code execution all within a single environment. Will discuss their training approach. Rather than programming specific tool usage patterns, they let the models discover the optimal strategies. Through reinforcement learning across thousands of virtual machines, they've created an agent that can work alongside you for hours, asking clarifying questions and accepting mid task corrections. Expanding the ways that we can interact with AI agents. The team shares fascinating challenges around safety guard, relevance around agent activities, and why things like date picking still remain mysteriously difficult for AI systems. They've revealed how small, focused teams are achieving breakthrough capabilities through careful data curation, suggesting that we're now entering a new phase of AI development where product insights matter just as much as compute power. Enjoy the show. Issa, Casey, Edward, thank you for joining us today.

Thank you so much for having us.

So you're the team behind the ChatGPT agent or agent mode. What is it?

Yeah. So this has been a collaboration between the former Deep Research and Operator teams. And we've created a new agent in ChatGPT that's able to carry out tasks that would take humans a long time. And we gave the agent access to a virtual computer. And through that it has two different ways to access the Internet, or actually more ways, but we'll get to that. It has a text browser which is similar to Deep Research tools, so it's able to efficiently access information online and search through things with this very, very fast text browsing tool. And then it also has a virtual browser which is similar to the operator tool. So it actually has full access to the graphical user interface, and it's able to click and type things into forms and scroll and drag and all these kinds of things. So together it's much more powerful than either of those two tools because one's more efficient and one's much more flexible. Flexible. And then we also gave it access to a terminal, so it's able to run code and analyze files and create artifacts for you, like spreadsheets or slides. We also, through the terminal, it's able to call APIs, so either public APIs or private APIs. If you sign in, it could access your GitHub or Google Drive, SharePoint, many other things. The cool thing about this tool is all of the tools have shared states. So it's similar to if you're using a computer, all of your different applications have access to the same file system and things like that. It's the same for the tool. So the model can do quite flexible things. And yeah, we'll talk more about this later, but I think it's just a very flexible way for the model to do very complex tasks on behalf of users.

Tell us a little about the origin story. How did this get started?

Well, our team worked on Operator, and.

Our team worked on Deep Research.

And so back in January, we released our first agent, Operator. This is a product that can do Internet tasks for you, like buy things on the Internet, shop for you, this kind of thing. And then two weeks later, we released.

Deep Research, which is a different model or a different product that's able to extensively browse the Internet and synthesize information, and it creates a long research report with citations for you.

And we were kind of thinking through our roadmap, and we were kind of like, hey, this is kind of a match made in heaven here. So Operator is really good at visual interacting with a web page, but it's less good at kind of the text browser reading long articles, whereas Deep Research is really good at reading long articles. But it has a tougher time with interactive elements or highly visual things because.

The tools are different. So Deep Research has a text browser, so it's able to really efficiently read information and search and synthesize information, but it's not able to scroll and click in the same way or fill out forms in the same way that Operator is because it has actually full access to the GUI browser. And as Casey was mentioning, Deep Research has some things that Operator doesn't have. And then similarly, one of the biggest requests for Deep Research is for the model to be able to access paywall sources or things that you have to pay A subscription for and Operator is able to do that.

And also one of our members of our team, Eric, he was running an analysis on the types of prompts that people were trying on Operator and we realized that it was a lot of Deep Research type tasks like research this trip for me, then book it. So it really is a natural combination.

In what way is 1 plus 1 equals 3?

So in the research we, we always wanted it to figure out how to let Deep Research has access to a real browser that can loading all the real contents that previous Deep Research cannot have access to.

It's funny that you bring up the one plus one equals three because not only did we combine Deep Research and Operator, but we also threw in a bunch of other tools that basically everything we could think of. So the terminal tool is there so it can run commands to do calculations. The Image Gen tool is a fun one. If it wants to spruce up its slides by making an image, it can do that.

Can cool.

APIs can produce PowerPoints.

Yes, it can do a lot of different things.

Tell us a little bit how people are using it. Knowing it's still early days.

I think the cool thing about it is we have some ideas of how we think people are going to use it, but I think we intentionally kept it quite open ended. It's called Agent. That's so vague partially because we are excited to see how people end up using it. So I think some of the things that we specifically trained it for were of course deep research type tasks. So things where you want a long report on a topic. Operator type tasks where you want it to do something for you like book something or book a flight, buy something for you. And then also tasks to make slide decks. We also spent a lot of effort on making spreadsheets and doing data analysis. But I think there are also just so many other things a model can do. So we're just excited to see how people use it. Kind of similarly to how when we launched Deep Research we saw a lot of people using it for code search, which was really surprising to us. We're hoping to see a lot of new use cases that we didn't even think of ourselves.

Would you guess it'd be more consumer or kind of B2B type use cases? Or is that a false dichotomy?

Hopefully both. Okay.

I think we're kind of aiming for the prosumer, someone who's willing to wait 30 minutes for a detailed report. But that can be in the consumer case or at your job. I think it could be good for both.

Do any of you have favorite things you've used it for?

For me, it's more like pulling data from our spreadsheet or Google Docs, like documenting our expander log and then make some slides to present the data or like, organize the data. It's pretty useful.

I've been doing a deep dive into ancient DNA. One of my, you know, one of my interests. Um, and there's actually a lot of exciting work going on. Like these past, like, five years, they're like, sequencing all this DNA and like, discovering all these facts about, like, oh, where did, like, this group of people come from? And like, you know, historical stuff. Um, the problem is that everything is so new that, you know, there isn't like a reference, like source material for like, to summarize, like a survey of these materials. But, you know, agent can go out and pull together all these sources and synthesize it into a report that I can read or slides that I can read. And I think it's kind of made for this topic.

Yeah, I like it for consumer use cases. Like, I've used it for online shopping, I think especially because a lot of websites require using visual browser because it will have a search filter or something that it needs to go through, or the model actually needs to be able to see what the item looks like. And then also for planning events, it's been pretty useful.

What's your favorite shopping query?

I think I was using it for clothes shopping.

Love it.

And you guys also showed us a really cool use case right before we filmed this episode. Do you want to share that one?

Yeah, sure. So that was actually something that one of our coworkers, Tejal, shared with us. She asked the agent to estimate OpenAI's valuation and based on things that it found online, create a financial model with projections to create a spreadsheet, also create a summary analysis, and then also create a slide deck presenting the results. And so hopefully the model is correct because it was. It had quite an ambitious projection for us.

It was an impressive slide deck.

Yeah, it was a good slide deck.

One thing I want to point out about this trajectory was that it reasoned for, I think, 28 minutes. And yeah, I think this is kind of opening up a new paradigm where you ask the agent for a task and then you step away and it comes back with a report. And yeah, I think as agents become more agentic, it'll be longer and longer tasks. And this is a good example of one.

Are these the longest running tasks you guys have have launched so far?

I would say so. Like I just did one that was an hour long and I don't think I've ever seen that.

I don't know how long codecs can run for.

That's true, yeah.

Is there anything special that goes into making an agent run for so long without kind of flying off the rails?

We have some tools to enable the model to be able to further extend its context lens beyond what's original, like the hard limit, so that the model is able to perform task by documenting what it's doing and step by step kind of increase the time it can do. The horizon of the task it can do without the human's interruption.

Yeah, it's also the flow to go back and forth between the model and the human also is very nice. So I can correct it as it's going, right?

Yeah. So this model is very flexible and collaborative and that was very important to us. So it's modeled after how you would interact with someone if you ask them to do a task for you. So imagine you're asking someone on Slack to do something for you. You'd probably give them instructions and then they'd ask you some questions and then maybe start doing the task. And then maybe in the middle of the task they'll say, oh, actually, can you clarify this to me? Or can you sign into this thing for me? Or am I allowed to do this for you? And similarly, you might remember something that you forgot to say when you first gave them the task, and you might want to interrupt them and just say, oh, hey, please also do this. Or you might want a status update if they're taking a long time to do it. Or you might want to redirect them if they're going on the wrong path. So that's what we modeled it after. And I think it's very important that the user and agent are both able to initiate communication with each other. So I think what we have now is probably the most basic version of what this could be. But it's better than anything we've released before in this area because at first the model can. Or the agent can ask you clarifying questions, similar to deep research, but it's more flexible, so it doesn't always ask you clarifying questions. And then you can interrupt the model so you can say, oh, can you summarize what you've done so far? Or oh, I forgot to say, I actually only want like blue sneakers. And then if the model is going to take some kind of destructive action, or if it needs you to log into something, it will also ask the user if it's allowed to do that.

Before doing anything on this topic, we kind of built this kind of computer interface. You guys saw it where you can kind of watch along with what the agent is doing and that actually persists for beyond the conversation. So like once it's done with the task you can actually go back and ask it follow up questions and ask it to like fix something or do another task. And you can also take over that computer. So you can click in and then now you have access to its environment and you can like click for it or like log in for it or insert your credit card information or things like that. And so yeah, I like to think of it as like looking over your coworker's shoulders and like, and like being able to take to take over if necessary.

Thank you for enabling the micromanager in me. Just kidding.

So we'd love to talk a little about how this works to the extent that you can share.

Yeah. So this agent is trained with the same technique as well on the reinforcement learning. So we give this agent model all the tools we have like implemented in the same virtual machine like text browser, like GUI browser terminal and the imaging tool. And then the model will try to solve the task like we curated like which are pretty hard task that the model has to complete with using these tools. And then kind of like we give we reward the model if the model completed the task efficiently and correctly. And for example after this training the model is able to learn to switch between these tools fluently. For example, for a task of you asking the model to research some restaurants and maybe booking a spot for you, we'll first do a deep research style text based browsing and then we'll probably also use GUI browser to view the like images of the food and also view the availability which is usually written in JavaScript that you have to use a real GUI browser. For example, if you ask me to create an artifact, it usually can pull sources from website and then use them in the terminal.

Yeah, I think the cool thing about this tool compared to tool use implementation in the past is that all of the tools have shared state. So it's like when you're using your computer and you have many different applications, you know if you download something it's going to be accessible to other applications. It's very similar. So the model can open a page in the text browser which is more efficient but then maybe it realizes it needs the visual browser so it can just seamlessly switch. Or it could download something using the browser and then in terminal it manipulates it or something like that. It can run something in Terminal and then open it in the browser. It's very flexible. And so it's just giving the model a more powerful way of interacting with the Internet and files in its file system and code and things like that.

And one interesting thing to emphasize is that we essentially give the model all these tools and then lock it in the room and then it experiments. We don't really tell it when to use what tool. It figures that out by itself. It's kind of almost magic, is the technique.

It sounds very similar to Deep Research. We had you on the podcast before. Should we think about this as the standard technique of how OpenAI thinks that agents will be trained going forward?

I think we can take this really far. Our teams haven't been collaborating for that long. We even framed this model run as kind of minimum shippable de risk. That was mostly for PR reasons internally, but this is really like the most basic version we could make together. And I think we have so much further we could push this with these methods. For example, the slides capability is a new capability. It's like very already impressive. It's a great work from a Aidan Paloma Martin, a bunch of other people. But there's so much further we can push that and improve using the same techniques. But I think we can take it further. But we probably need other things too.

Yeah, I feel so far it's pretty magical. The same IO algorithm just works like o reasoning, like a Deep Research with toolkit and then now a more advanced computer use browser use agents.

Where does it run into the limits with this strategy and. And with this model specifically as well.

I think the interesting thing with this model is that because it's able to take actions with external side effects, there's a lot more risk. So for Deep Research, it was read only. So there's kind of a limit to what the model could do in terms of data exfiltration and other things. But with this, in theory, the model could successfully complete a task but take a lot of harmful actions along the way. Like you could ask it to buy you something and it decides to buy just like 100 different options to make sure that you're satisfied. Exactly. Or you can think of many examples like that. So I think that safety and safety training and mitigations was kind of one of the really core parts of our process with this model. And yeah, maybe Casey can talk more about it.

I was going to mention that kind of along the same lines. It's like this contact with the Real world that makes things difficult. We have to train this on a bunch of VMs. It's thousands of VMs maybe and things break and as soon as you're hitting a real website, the website's down. Or you're hitting all these capacity limits and load testing and this kind of thing. Yeah, it's really the very beginning and we're going to iron out all these details and continue. But that's a major limitation.

How do you think about, from the safety perspective, building in the right guardrails and how do I make sure the model's not logging into my bank account and sending it all off to a Nigerian prince?

Yeah, that's a very good question. Yeah, this is definitely an emerging risk where, you know, the Internet's a scary place. You know, there are a lot of like attackers and like scammers and this kind of thing. Phishing attacks like this goes on and on. And yeah, our model is a bit like it can, it can reason about these things, like if you tell it to be careful. You know, we've done some safety training to make this like more robust, but sometimes it can get fooled and sometimes it, you know, is a bit too overeager to complete your task. Um, we have a long list of mitigations and the team has worked really hard to like stack together a bunch of techniques to really try to make the model as safe as possible. Um, so, you know, one example that I'll call out is that we have a monitor that looks, kind of looks over its shoulder and just sees if anything looks funny, like whether it's going on a weird website or anything like this kind of like antivirus for your computer. It's like just kind of persistently watching and then if it looks like there's anything suspicious, then it'll stop the trajectory and stop there. Of course we can't catch everything and this is a major area that we'll continue to iterate on. We do have a protocol for if there are new attacks in the wild that we discover or we encounter, then we can rapidly respond and update these monitors. Kind of like you would update your antivirus software. It would kind of pick up on these new attacks and hopefully keep you safe.

Yeah, I think the cool thing about the safety training is that it's been a really cross org effort from the safety team, governance team, legal team, research team, engineering team, like so many others, and we have so many mitigations at every single level. We did a lot of external red teaming, internal red teaming, but yeah, as Casey mentioned, there's more surely when we release the model, there will be new things we uncover. So we just need to make sure we also have robust ways of detecting those and then mitigating those.

For some of these models. There's a risk of what you can do with the models, whether it's creating biohazards or otherwise. How do you guys manage some of that?

Yeah, it's actually bio has been heavily on our mind. Yeah, the team has been really thoughtful about. Yeah, this agent, we think it's very powerful. It can do research, it can really speed up your work. But that also means that it could speed up harm. And kind of one of the top things that our team has been looking into is the risk of biological biorisk. So like creating bioweapons, this kind of thing. And yeah, the team has been really thoughtful about how to mitigate against this and generally being very cautious. We did like many weeks of red teaming to make sure that this model could not be used for those harms. And a bunch of other mitigations in place. Shout out Karen, who spearheaded this effort. And yeah, in general, I think we're. We're very aware of this and just trying to be very cautious.

Yeah, makes sense. Tell us a little about the team that came together to build this.

So as Casey mentioned earlier, we had deep research research team and then deep research Applied team and operator research team or computer using agent research team and operator applied team. And we effectively merged everybody. We all work really closely together, both the research team and the applied team.

And the vibes have been great.

It's been so fun.

Isa and I have been friends for a long time, so it was a natural.

Yeah, it was really fun.

How many of you are there on.

Deep research for the majority of the time? Three or four. Now we have some new people, which is very exciting. And then on kua.

On kua, I think around six to eight, somewhere around there on the research side.

And then we have an amazing applied team. So like engineering product design led by Yash Kumar. And then he has just a really cracked engineering team. So it's been very fun to work really closely. I think that's one thing that has made this collaboration really special is that the research and applied teams work so closely. And even from the beginning when we're defining what the product should be able to do is it's very much a collaboration between research and product and design. So we go backwards from the use cases we want to be able to solve to training the model and building the product. And obviously it's not able to do all of those things fully yet. And it can do some things that we didn't plan. But I think it's a good framework for us when we're starting a project. It's very grounded in how we want people to use it in the real world.

That's a way smaller team than I was expecting. Small teams can do amazing things.

Yes. You've built a lot.

Yeah. And we haven't been working together for very long.

It's been a few months.

Yeah.

And actually the boundary between the research team and the applied team are not very decisive because during the model training a lot of applied engineers, they are helping us training the model. And also after we change the model, some research team members are also working on the like setup as model deploy the model to the real users.

What was the hardest part about training this agent?

Yeah, I think one of the biggest challenges we have is how to make training stable. Especially given when we train deep research, it's only using browsing and Python. It's pretty mature tools there. We've been using it for a while. But when training agent model it have some new tools like computer and also the terminal like a bundled in the same container in the same virtual machine as a computer. So it's actually quite hard to train it because we are literally set up hundreds of thousands of virtual machines at the same time and they all visit the Internet and we. So it's one of the biggest challenges we see that actually the training sometimes will fail. But finally we are very happy that we get this model trained.

So VMs, VMs, all back to the engineering.

Tell us about what's next. More sources, more tools, better model. How do you think about it?

Well, I think one thing I like about our agent framing is that you can ask it to do whatever you want and you know, you can ask it to do like every possible task you can imagine. It just might not do it.

Well, could you tell it like go make me money on the Internet.

You can tell it that it will try. It'll try.

Should we try that right after this?

Let's do it. But yeah, I think it's really a matter of like improving the accuracy like the performance of task of the whole.

Distribution of tasks that anyone does on a computer.

Right.

Which is a lot of tasks.

And through this iterative deployment we are very excited to see what's the new capabilities that our user will find in our agent. Like the coding ability in deep Research or deep research ability in Operator.

You were using the agent mode for.

Coding Yeah, I use it for coding a lot because I feel it's actually not very always try to rewrite my whole code base. It just actually have some small editing and also it actually read the original docs of different functions pretty well. So I feel it hallucinates less on the function coding.

Oh, interesting. How do you choose when to go to codecs versus when to go to agent? For that?

For the agent I use it more similarly to how I use 4:03. So it's more like an interactive experience. For the codecs it's more like you have some well designed problem that you want a coworker to solve and then it will make a PR for you. But for the engine it's more like just give you a function or give you a suggestion.

Cool. And it can do code search because it can access GitHub through the API connector. So code search kind of things.

Yeah, it almost feels like the agent roadmap. Up until now you've built the different almost like appendages of what it would take to have an agent. And by combining them all, this really is like the first fully embodied agent on a computer. I think it's very exciting.

Yeah, I think another area that we're excited to push on is the experience of collaborating with the agent. I think this model is actually very good at multi turn conversations and it's very nice to continue working on a task with. I think that's one of the deficiencies of deep research. A lot of people will do multiple deep research requests in a single conversation, but it doesn't always work so well. So I think we're really happy with this model's multi turnability and we just want to improve even further. And then I also think like personalization and memory for agents will also be very important. And also right now every agent task is initiated by the user, but in future it should also be doing things for you without you having to even ask in the first place.

Yeah. I'm also pretty excited about the UI and UX surrounding the agent because right now I think obviously we're working in a chatgpt world. It's like you start a conversation and it goes. But, but you can imagine a lot of different modes of interacting with an agent and I'm really excited to explore different ways of. Yeah, different ways of interacting with the agent.

Do you see this as always being a kind of single omniscient super agent or will there be the financial analyst sub agent and the personal party planner sub agent? What's your vision for how that Kind of plays out.

I think people have different opinions on this. I think in the limit, if you could just ask one thing and it can figure out what it needs to do to finish the thing that you want it to do for you. That seems like it would be easiest like if you just had a really amazing chief of staff who knows how to route things correctly and basically can do anything you need. That seems like it would be pretty easy.

I think I agree with that take and like even in some of our trajectories where like, I don't know, you're asking about, I don't know, maybe like a shopping task, like sometimes it'll go into terminal and like do some like calculations, you know, budget and like I think the model should be free to use all the tools at once. It doesn't need to be a financial analyst to like have the financial analyst like tool set.

Yeah. I feel like when you launch the product, it's sometimes makes sense to have some GPTs like a customized model or customized instruction to put the model into a specific role. But in general, when training the model, there are lots of positive transfer between deep search cooperations, also slice generation. All of these skills are transferable. So it makes much more sense to just have a single agent like, you know, as an underlying base model.

Totally.

I guess even though, you know, people do different types of work, we're all fundamentally, we're sending emails, we're making slide decks, we're doing a lot of the same work in front of a computer. I'd love to understand some of the learnings from the reinforcement learning perspective. Like it seems like that's the method that seems to really be working for you guys with agents, was it like very data intensive to kind of get to this point of having an agent that's so good at, you know, such a wide variety of tasks or like what, what were some of the learnings from an RL perspective?

Yeah. So you know, we actually, we create a bunch of very diverse set of tasks, like some tasks to find some, you know, like a very niche like topic or very niche answer in the Internet or some tasks, you know, like, like just very similar to deep research, like you need to write a whole fully lensed article and also lots of tasks like just all the tasks that we want the model to be good at. So far we think that as long as you can grade this task after the model give you a result, you can judge whether the robots or the model's performance is good or not, you can kind of reliably Train the model to be even better on this task.

Was there anything special you need to do to make sure it had good turn by turn interaction with users when doing that training or was it just about the type of trajectories you collected?

Yeah, so I think most of the time we focus on end to end performance from a way of specify the prompt, how to complete the task and.

Somehow it's very good at working with users.

To your question, the reinforcement learning is very data efficient. So that means that we're able to curate a much smaller set of very high quality data. The scale of the data just so minuscule compared to the scale of pre training data. So we're able to teach the model new capabilities by just curating these much smaller high quality data sets.

I will say to get the operator piece to work. Well, um, you know, before we have, before we do rl, the model has to be good enough to like have like a basic, like basic completion of tasks. Um, and our team has spent a lot of time in the past, like over the past two, maybe three years getting the model to that point where it's able to actually reason about a page and like kind of like understand the visual elements really well. So this model is built on all that as well.

Actually, could you say a little bit more about that? Because I remember early days of OpenAI, this was always part of the world of bit stuff and you're trying to RL the mouse paths and it was just way too unbounded of a problem. What's changed now for that to be kind of solvable?

Yeah, that's great that you point out the world of bits. This project does have a very long lineage dating back to 2017 or so. Actually our code name is World of Bits 2 for the computer use part.

That's awesome.

And yeah, what's changed? I think essentially the scale of the training has changed. We have, I don't know the multiplier, but it must be like 100,000x or something in terms of compute the amount of training data. We've done both in pre training and rl. So yeah, I really think it's just scale and the scale catching up to our ambition, I guess.

Wow. Scale is all you need.

I believe it.

And some good data.

Are there particular capabilities or functionality that you're especially excited about in agent mode?

Yeah. So this model is actually pretty good at doing some real research like data science and also summarize the reports or the findings in a spreadsheet. So we have some evaluation like data science, bench we evaluate the model and it's actually outperforming the human baseline. So in some sense, it's actually superhuman. In some research tasks, we can rely on the model to perform some basic analysis for us.

And this is an area that John Blackman on our team was really pushing on spreadsheets and data science. So shout out John.

Spreadsheets and data science. You are automating us out of a job over here. Elevating us out of a job.

Enhancing.

Another thing I'm excited about is we released Operator in January and it was decent at clicking around, but I think we've substantially improved that capability where it's much more accurate and just kind of getting the basic things right is what I'm actually excited about. Where it can reliably fill out a form and, you know, do those kind of things.

Date picking.

Date picking. Date picking still needs a bit of.

Work, but for some reason date picking is just the hottest task. It's hard for human too, like picking a date in the.

The calendar dropdowns.

Okay, last question. It seems like you guys have the overall framework and structure in place for something really interesting here. What's ahead? Where do you go from here?

I think the thing that we're really excited about is that this tool that we've given the model access to is very general. It's basically most of what you could do on a computer. And if you think about all of the tasks that a human can do on a computer, it's very extensive. And so now we kind of feel like it's a matter of us making the model good at all those tasks too, and figuring out a way of training on as diverse of tasks as possible with this very general tool. So I think there's a lot of hard work ahead of us, but we're very excited about it. I think we're also excited about pushing different forms, ways of interacting with the agent. I think there'll be a lot of new interaction paradigms between users and virtual assistants or agents. So a lot of exciting times ahead.

I can't wait to see it. Thank you.

Thanks for joining us. Congratulations on the launch.

Thank you so much.

Thank you for having us.

Sa.