In this conversation, Robert Ta and a guest discuss OpenAI's newly announced Agents API with "computer use," which lets AI agents operate inside an OpenAI‑hosted browser to perform multi‑step tasks such as signing into services. They highlight how the API can manage sign‑in flows by prompting the user locally (phone or desktop) and securely passing credentials to the cloud, solving a major pain point of current cloud‑based automation that struggles with authentication. The dialogue then expands to why "computer use" is essential today because many real‑world services lack APIs, requiring clicks, scrolling, and form filling—essentially making the AI act like a human. They speculate that in a decade the UI may evolve to be fully agent‑friendly, eliminating the need for such interaction. The conversation also touches on user adoption gaps: most people aren’t paying for AI services, leading to divergent experiences and perceptions of AI quality. Overall, the exchange frames the current necessity of button‑clicking AI, the technical solution offered by the Agents API, and broader implications for future UI design and user expectations.
Chapters
One thing I just saw that I think might be interesting also, this was also announced today, the Agents API with computer use.
Let me go to it. All right. Computer use. Let an agent complete tasks in an OpenAI hosted browser.
Yeah. And the thing that I think is really key about this is to handle each website access request. See that line? It was like number 3. Yes. So like if the task needs an account, it handles sign-in. So there's a way, you know how what's like really annoying about cloud? tasks is like, if it needs to log into something or it needs to do something, it's kind of a pain in the ass.
It's so annoying. Yeah.
And that's like one of the reasons I haven't used cloud that much. But now I'm starting to see this UX where you have the cloud thing running. The moment it needs you to log in, it will push something to you locally and have you do the login, like maybe on your phone, maybe on your desktop. And then it will figure out how to send those credentials to the cloud. And this API allows you to do that. Like it has a handle sign-in. You want to click the handle sign-in, you can see like how it does it. It's like, okay, tasks such as reading issues in a private GitHub repo require an authenticated browser. Your application handles sign-in so users can choose a login method and enter credentials outside the chat. So sign-in can involve several requests. You know, it might ask the user for an email, enter an email address, then enter a verification code, et cetera. So, you know, like, yeah, this is a way for you to like I mean, I think they're already doing this in their cloud. I haven't tested it, but I've seen some videos of it. This allows you to do the same thing.
Yeah, that's really cool.
I use computer use. I mean, the reason why I'm so stuck on Codex is computer use.
Wait, okay, say more on that. What are the top use cases you use computer use for with Codex?
There's a lot of things out there in the world that don't have APIs. You know, you have to click on something and click on some buttons to do something, whatever. And computer use is like basically how you can have AI act like a human. You know, maybe 10 years from now you won't need computer use because we'll just collapse all the user interface to be all agent-friendly. And the idea of computer use doesn't have to be a thing. But for now, yeah, it's just the world is very jagged. Yeah. In, in that you need to like click a button and scroll and do all this nonsense.
I love how you say it. We're such in a bubble where we're like, all this is— this clicking stuff is for the past humans and they're not doing that anymore. If you talk to somebody that's not in SF or like New York, I mean, I know you're in Portland, but you're very connected to SF. These people are like, how do I use AI? And that's where Muse and like Dots, I think, are meeting those users, or Grokbot.
Potentially. I mean, I would say there's look at the statistics, right? Like most people are not even paying for AI, which blows my mind. 'Cause if you're not paying for AI, your experience of AI is completely different.
Yes, totally.
You might think it's not that good. You're like, uh, what is it? Like, I tried this thing, it didn't work. Oh yeah, you're not using like anything modern.
Yes.
Per the standards of AI. So yeah, I think. You're right. Like, people's perception of AI is totally different than what—
than ours, than our
experience.
We have a very different experience. We're like, use computer use because I can't be fucked to go into that anymore. They don't have an API for this? Go reverse engineer the API. And it's like, your typical mom, like, taking their kids to, like, soccer practice is not thinking about this. They're like, can you just, can somebody just help me save time on literally anything? Because my day is so chaotic.
