Where AI products go next: voice, agents, and self-driving software | Tara Sesha and Nan Yu (OpenAI)

All right, we're here, and I warned them backstage what we were going to start this talk about because I know there's a lot of questions about OpenAI, there's a lot of questions about the future of product development, but what I want to talk about is—

What do you want to talk about, Claire?

The toggle. I want to talk about the fact that every AI product has 2 faces. It has left toggle and right toggle. And there's been a lot of conversation today about iterative product development and shipping things that are imperfect and getting things into the wild. And so Tara, I just want you to talk to me about shipping imperfect things as a toggle and how do you make these decisions about putting things in the wild that, you know, may not be the perfect thing right now, but are the right thing ultimately to build the right product?

For sure. I will say this to start, like, you know, I'm the daughter of Asian parents. I'm not used to doing imperfect things. You know, you have to output something perfect every time. I also worked at Stripe for like 7-some-odd years, and everything at Stripe is like deeply polished and deeply considered. And did you think through it 100 times? And did you name this enum the exact correct thing? And I think what has been a big change about shipping things in this era is actually urgency really matters.

Yep.

In terms of being get— getting the functionality out there. And also, there's a lot of theorizing that one can do before you ship something, but nothing compares to the actual empirical evidence of seeing users try it, use it, and then iterating from there. So I've really had to shift my mindset from how do you get it perfect a priori versus how do you ship something and iterate on it as quickly and effectively as possible. With the toggle, is it the perfect solution? No, it is an imperfect solution. It is certainly not the ideal thing to ship, but what was really important was getting this agentic harness or the ability to use an agent to get things done in the hands of the over billion users that use ChatGPT. We needed to ship something that allowed us to do that without disrupting developers' workflows. As such, an imperfect solution.

And you know what? I do believe that in a year, maybe we won't be having the same conversation about the struggle.

For sure.

And Nan, we've had this, we've had this chat before where we've talked about right now you have to build and throw away a lot of stuff, whether it's code or product. And I'm curious how you think about deciding what to deprecate and what to keep around and keep alive.

The toggle's a good example of this where I think there's an obvious next step, which is no toggle. There's a story that evolves from step A to step B, and if you can take users on the journey with you, then they're a lot more accepting of the change and the turn and the product evolution, especially if they appreciate that they can have an understanding and some transparency over what's actually going on behind the scenes. I think as long as you have a coherent story that users can follow along with and with the amount of attention that they're paying, there's actually a lot that you can do and not really burn any kind of bridges with them.

Yeah, and I'm curious, the toggle is maybe imperfect, and we may build a lot of stuff that we have to throw away, but you do exercise good judgment, and you do exercise caution and reason inside the products that you've worked on or are about to work on. What constraints do you hold really strongly when building product that's moving so fast? What keeps that quality bar high for you?

I think there's a few different elements of it. One is certainly, are we doing something additive for users? Like, are we unlocking additional value? Is there something genuinely useful in this product? I think the second is that we have this internal bar of taking a product, shipping it internally, trying it out, having everyone use it. If there isn't enough uptake internally, and that's not a specific number or exact bar, but there is, is it retaining well? Do people find something delightful about it or something surprisingly great about it? Is it unlocking a new use case of the models? Is it doing something novel? I think that sense for this is adding to their experience? This is something that is unlocking a new set of functionality. And then probably most importantly in this era, is this aiming for 2 to 3 months of where the models will be rather than being overly anchored in the present or being overly futuristic and unusable? Is this at that perfect zone of being, I don't know, sufficiently AGI-pilled? That, I think, is the bar here. Is it valuable? Is it Do users care about it? Is it surprisingly great for users? Is it in line with model capabilities? Is it getting out of the way of the model doing a great job?

Nan, what about you? What constraints do you think help you build great product?

The biggest constraint is honestly people's understanding and ability to absorb what you're giving them. We talk a lot about capability overhang, but models can do so much and people aren't necessarily taking advantage of all those abilities. I think that's where the gap is. It doesn't matter what you put into it. If the ability for people to absorb those changes and absorb that capability isn't there, then that's the real bounding process that you're dealing with.

We're talking right now, you two are working on Codex and Work, and there's a lot of business users, including myself, that have built businesses around relying on this tool. I think I've spent a minute and a half in enterprise. And a lot of people say that B2B customers cannot absorb change. They, I mean, like customer success people tell me they can't absorb change. Salespeople tell me salespeople can't absorb change. Like we cannot absorb the rate of change. Do you feel like that still stands true in the AI era or do you feel like that statement is, is broken right now?

Some of it is certainly true in that I think the current pace probably to most enterprises feels beyond breakneck, like something beyond breakneck, like a Lollapalooza of features and advancements. Like, it certainly feels maybe to some overwhelming. But at the same time, like, if you're not shipping the latest and the frontier to these enterprises, you'll end up getting leapfrogged. So, a great example of this is that most of our enterprises were using chat So they were merely asking questions and getting answers. Whilst that was happening, the agent revolution was going on, and we needed to get agents to these enterprises as fast as possible, otherwise they might consider alternative products. While they have their own, maybe in the day-to-day pace of absorbing changes, we needed to ship this giant revolutionary change that broke all of their processes and broke all of their conceptions about how enterprise businesses should receive new updates so that they could actually unlock more value. And so it's one of those things, I mean, it's like a truism of building product, which is like, you know, do what your users need, not what your users say they need. And I think that applies here almost more than it ever has before.

I wanna talk about agents since you brought them up, because I do think we're in the age of agents, and I have certainly been in the age of agents for a little bit. And I want to have a debate with you.

Oh, gosh.

Which is— or I want your opinion. Maybe we won't have a debate. Maybe we'll agree. Single identity agents versus multi-identity agents. My beloved Codex docked on my computer that I talk to all day versus my 40 Groq bots that are load-bearing across different parts of my business. I do think as product builders, people who are building agents, we're seeing this divergence of Do we centralize on a central generic agent identity that can do anything, let the models get— get out of the way of the models, or do we artisanally craft sole MDs and have little micro agents for the job? I'm just curious, coming in fresh eyes, where do you land, if anywhere, on that spectrum from a design principles perspective?

I think when designing this, the most successful efforts have really tried to map onto human nature. We have millions of years of evolution that help us understand how to hold different things in our heads. 40 agents is quite a lot. If you asked anyone, would you manage 40 direct employees? Jensen Huang might do it, but a lot of people would look at that and be like, I don't know if I could actually keep up with that many threads ongoing. I think there's something in there where we need to group these things into bundles of activity. If you look at the— you ask people, hey, how do you manage your agents? Usually what you'll hear is, well, first I have this chief of staff agent that manages all the other ones for me. It's like, OK, well, you've just cheated. You're not managing 7. You're managing 1 that's running your team. I think that that dynamic immediately comes to the front as soon as you let people do this kind of stuff.

What about you? What do you think?

You know, my mind with this, like, certainly there's a philosophical element to it of like, you know, do you have this one unified entity? Do you have these like smaller entities? But actually there's a lot of really tactical questions that go into it. Like, how do you think about data access and permissions? Like, if you have one entity, is it like switching between using like a service account or your account? Like, how do you manifest that to the user in an easy way to understand? If you have a single agent, like, does it have segmented memory? Like, every time it joins a process, private Slack channel, is it almost a different agent? Is it like severed in the Slack channel in like the Severance sense? Actually, to me, answering this question goes back to really specifically the use case and probably thinking through all of those edge cases of like how should the memory behave? How should it talk to each person? Should it be using their credentials or should it be using a universal set of credentials? It's actually like there are philosophical things one could have and opine on in this topic, But to me, there are just so many practical product considerations. Like, the use case matters so much in deciding this.

What I love about these 2 answers is I think for product folks, everybody in the audience, plus designers, there are these 2 sides of the craft that we do. There's this very human side where you're trying to like— there's inception and severance, I guess. There's like the inception of like, how do I get in the mind of a user and what are their mental models? And How do they interface with the world, and what is the most natural way for our products to touch their problems and solve them? And then there is systems and architect thinking, which is in order to make this manifest this into the world in a realistic way, I need to think through all these systems, all these platform components, all these edge cases to make sure that comprehensively we can even deliver this. to the user in a sensible way. Do you think those are the 2 skill sets you need to design and build great agents in this moment? Are there other hard skills you feel like you're relying on now more than ever as you design or think about agentic experiences?

You're describing systems thinking plus user empathy. These are very classic product management types of ideas that we applied to a different universe of technology in the before times. But now, We have to start from scratch and from those principles rethink how do these products want to be shaped. I think those are very important, but ultimately they're just the old principles in disguise.

I would maybe add a thing to that, which is, again, I mentioned I used to work at Stripe, which is a very academic company, and my parents are very academic, and I love thinking about those things in abstract, but I'd say the third thing that you just really need in addition to user empathy and systems thinking It's like a relentlessness to try it out and keep iterating, and you're going to take a lot of pain while you do that. There's just an element of relentlessly trying and trying and trying and trying and trying to see what works. The ability to learn maybe from those feedback loops that certainly has always been necessary, but just the clock speed has gone way—

Yeah. You're throwing a lot of spaghetti against a lot of walls all the time. Actually, your chief of staff agent is telling all the other agents to fling spaghetti at the walls. You know, talking about systems thinking and platform, I am really curious because all of us are looking to tools, certainly ones that you build for productivity and efficiency. And I was telling somebody backstage, I just love computer use now. Like, don't even give me an MCP. I wanna point computer use at your UI and then, you know, I don't care the, the token. Congratulations on me selling me tokens. But, You know, I think about what is the intersection between the tools we had and the tools we have now, what you build, what you platform for. I'm curious how you just think about the broader ecosystem as you're trying to enable productivity for people at work.

So I think we think a lot about platforms and ChatGPT as a platform and what is the functionality that like natively goes into the platform. What is functionality that like we might allow folks to build in, whether that's like first party or third party as as plugins or additional interfaces on top of it. That's how that relates to the ecosystem. In that platform, what are the hooks that we need to expose to a third-party meetings app so that it can be most effective for you in Codex, using Codex, using computer use, using all these tools? But I don't think it's so binary as what should be in our platform or in our productivity system versus in one of these apps. should offer an array of different tools, like for perhaps like a plugin to be able to like get all your meetings data and understand, you know, what you need to do next. But if that doesn't work and those systems don't interface well, then there's always computer use as like a next layer or as like a fallback to be like, well, if the user wants to get this task done and, you know, the prebuilt tools or like ecosystem tools aren't available for it, then there's like always something else that they can fall back on. And so I think it's more about structuring these capabilities in the platform in terms of layers, exposing them as like a set of hooks that third-party developers and first-party developers can like hook into and build cohesive experiences, having the right system for those experiences to be composable with one another. And then ultimately all of this is just serving the user's goals of as they try to get their tasks done, if like the first layer doesn't work, there's like a second layer for them to be able to accomplish their task as you like traverse all those capabilities.

What do you think about integrated experiences, like do it all in one platform, let's live in our house, versus expanding into this ecosystem? And how do you build— I mean, the thing that I was thinking as you were talking is you have a non-deterministic model on top of a natural language interface functionally with a computer that can click and do anything with third-party tools and integrations you don't control, and you're still trying to build a comprehensive, high-quality, good, branded, beloved experience? Like, how do you even climb that mountain as a product leader? Do you have any— what genius are you going to bring in to help us here?

Look, I don't know if this is a new or unique idea, but there is a huge difference between getting, like, all of the job done versus getting everything except for the last mile. The last mile honestly feels sometimes worse than just like, well, it's a non-starter. If it's a non-starter, you can just like, look, I'm just going to do it myself, or I'll try another route, or whatever it is. But if it gets like 99% there, then it just barfs, then you're like, well, this is actually a worse unfulfilled promise. I think what feels great about computer use a lot of times is that it will always work. It might be a little bit slow, or it might cost a lot of tokens like you were talking about, but it will get the job done all the way. As the ecosystem We wait for it to catch up and to provide the right hooks and the right MCPs and all that stuff. It will 100% do the job for you. I think that ultimately, that's what makes it feel super great.

I think the right experience amidst all those complexities and changing things, the ecosystem, the interface, the model itself, the tools like computer use, the unpredictability of all of it, is that the user should almost feel like the Charles Eames quote that your user should almost feel like a guest in your home. like you've anticipated all the things that they want to do, and you've provided for them gently. So should our user in our app feel like we've anticipated their needs. And in some ways, it's like the model increasingly doing that.

Well, I have to tell you, every time I go into your home, it tells me it's time to update.

Yeah, yeah, yeah, yeah. Get the latest build.

That is the welcome mat that I get. It's like I walk in, it's like, you want a reno? I got a reno for you. Well, I'm curious. You're designing this product while it's flying off the shelf. You know, a lot of the things, Nan, as you said, are old things, user empathy, systems thinking, but there are some new things, right? There are some new skills. And Tara, I'm curious about working with research because this is something I don't think a lot of folks in the audience have done in their careers because I think it's very new for product people or product leaders. What have you learned from going into this new role where you really have to collaborate with research?

Yeah, I will say for me, working with research was entirely new coming to OpenAI, so I'm certainly not the expert on this topic. I will give you the novice's view, like Connecticut Yankee and King Arthur's Court sort of view of this. My take coming in working with research was, oh, working with research is different than working with engineering. That might be obvious to everyone here, but I was like, oh, it is quite different. Inasmuch as I can add something to working with research, it is for me to come with very, very specific use cases, a really clear idea of what users want to do and their goals, to have done sample readings, so understand literally what was the session and what did it look like and why did the model not do what it needed to do. If I can write evals, that is like— I'm there, learning how to write good evals and writing evals as much as possible is the most important thing. Inasmuch as I can say, with this prompt and these skills, I was able to achieve the outcome with the model, that is the ideal thing I can take to post-training and be like, look, now how can we post-train this capability into the model from the beginning? I think it starts with first understanding that this is a wholly different way of working that I really didn't know how to do before. And then getting to the point where you're good enough at bringing the specific user data, and then if they trust you sufficiently, you can then write the eval to get that iteration loop going.

How far in the future do you feel like you live as a product leader? Are you— because I do think there's this time horizon of I could fix what's right in front of my face, and there's stuff to fix. There's I know what's coming next, and I can anticipate where the product's going. And then there's like, the models next year are going to be out here. Where do you feel like you have to live as a leader building on top of that foundation that's moving?

You know, as soon as you said that, like, do you live in the future? I was like, I live in the now. Like, I don't live in the future. I live in the now. No, I do. I think I have to live a little bit in the future. To me, it's like the ideal is 2 to 3 months. Can't be too big-brained and be like, this is the future in years, because I think that Inevitably, our predictions of that are almost always wrong. I have a personal doc of incorrectly predicted things that I thought were going to happen and didn't happen, just to remind myself of my inability to predict the future years out. Then at the same time, if you build exactly for today, you're going to get left behind. I constantly repeat this, but it's like 2 to 3 months out.

I do want to just— let's poll the audience again. It's the end of the day. Let's get interactive. Who right now is doing 2027 annual planning? And who just heard her say you can't go out more than 60 or 90 days? I think this is the fundamental tension. It's so hard to even guess.

But it's not the same for every business. Again, just to use an example I'm familiar with, Stripe's business is very different than OpenAI's business. The dynamics of the payments market look largely similar to how they looked before, certainly accelerated in other ways. But you know the forces, and you can kind of model, like, oh, here's the bull case, bear case, base case in a way that for OpenAI's business, you just cannot do. Well, maybe Sarah Friar can, and she's the expert, and I'm not as good as her for sure. But it's hard for me to predict OpenAI's business the way I can feel like, oh, I understand the levers in Stripe. Maybe it is appropriate to plan many, many months out, but I think it involves understanding the pace of the market you're in and understanding, given the pace and the dynamics of that market, at what cadence should I be thinking and planning in the future?

So, what you're hearing is Stripe PMs still have to do their annual planning.

Oh, for sure. Good luck to you all. Like, yeah, for sure.

All right, let's talk a little bit. Let's close out with then and now. So, I want, you know, we've all been in product for a minute and a half as fresh 26-year-olds. What still matters from a classic PM craft, and what do you think is newly table stakes?

I think the emphasis changes for a lot of reasons, but we talked about just absorption rate and capability overhang. I think something that probably matters way more now than it's done before is onboarding. I think we've talked about— people have talked about onboarding as an important thing, but it's also something where it's a little bit specific and a lot of engineers don't love working on it and things like that. But I think really to get a lot of success, thinking about what is the initial experience, how do you get people to really understand what your offering is constantly is probably something that matters way more now than it does before.

Tara, what do you think?

Yeah, I think users understanding privacy, their data, how that's used is way more important than it always has been. Users understanding norms of your product, like before any sort of unpredictable behavior you could get away with because there was some like settings menu or some popup that could appear. But now if the, you know, agent is doing things semi-autonomously, like you actually really need to make sure that at any given point a user can like predict what is probably going to happen with a feature or a primitive that you're introducing. And in terms of like just product thinking or getting up and running as like a PM, In some ways, it's going to sound like a really boring answer, but all the usual stuff still applies except the importance of testing things empirically and moving faster has only increased. It's like, yeah, do all the other stuff you used to do, which is get in the details, have the high level and the low level at the same time, really understand the design, really understand the product, but also do it all faster and test with users. It's just become way, way more important.

I have one more that Non is going to become a victim of when you are in the OpenAI Slack, which is, The OpenAI product and engineering team is highly DMable, highly accessible, highly visible. I mean, they're getting like feedback, UUIDs from me on the regular. Like, I didn't like what Astra said. It had an attitude. Like, and so I'm curious, you know, certainly I have tried to do this in my own life, but do you feel like direct user relationships right now are becoming table stakes for product leaders? Like this idea of the DMable PM, I think is one that I see a lot out of OpenAI, and I'm curious, like staying close to the ground, if you feel like that's becoming more and more important.

My background is in like developers and enterprise, so I've always been a DMable PM in the sense that people could always message me and ask for features. Um, but I think what kind of feels new is that for like a large consumer product or a product that everybody uses, to do the same level of accessibility. So in some ways, like this, it doesn't feel abnormal to me. Like, of course you should be in direct contact with your users. And as I mentioned earlier, like, your value to research is like knowing specifically what the users want and what their use cases are. And so the more you can be a DMable PM or engineer, the better. Like, if you didn't like Astra's output, I was actually thinking like, hmm, Claire should send me those outputs. Like, it would be really interesting to like know what's happening there.

I will.

Like, that stuff is like the important thing and part that's like how you add value. But I'm curious for Nan's take.

Yeah, I, I, I think, you know, you've worked at, uh, like dev tools, right? And, you know, engineers are extremely picky. They're extremely detail-oriented. Often you're not going to get good feedback until the second or third follow-up question. And I think that's like what a lot of, uh, PMs are feeling like a change about, right? Because if you're designing an agentic experience with like natural language interfaces, the problems that people encounter, like Astro gave me some attitude, like they're pretty subtle. That bears some explaining. What exactly did it say? What was the context? Did you make it mad? All sorts of follow-up questions that you need to have. I think having a more direct, deeper relationship bears a lot of fruit there that maybe you wouldn't have been able to take advantage of before.

Neither of you have gotten my DMs, so I'm very excited for the next 90 days. for you all. We'll put us in a group chat. It'll be really fun.

Great.

Okay. Last question. So when Dan was up here with the Anthropic team, he asked, like, you know, what in a year about the product craft, about the role will be different? I want to talk about the stuff. So, like, what do you feel is going to be different about products? Like, what we touch, what the buttons on the site, the experiences, like, where do you feel like this is all going? Because I feel like every week, every month we come up with a new form factor, a new thing that matters. Just curious, as product thinkers, what do you think is going to be big in 2027?

I have an answer for you, and it's voice.

Voice.

I love voice. You should try out GPT Live. I love voice. Like, we're yappers.

Yappers. Hillary Bradley, the Yappers API. Genius.

Voice has changed the way I work. I just want to talk to computer. Like, I do not— like, it's so much better. It feels so much more natural in some ways. Like, I'm sure all of you have been tech support for your parents. Like, the existence of voice has reduced the amount of, like, tech support I've had to do for my family to, like, an incredible degree. Like, I was onboarding our own people team at OpenAI onto a new product, and the existence of voice, like, changed the game. Like, it is such an intuitive interface for so many people. It is like the natural way they interact. The thing Nan said earlier about, like, you kind of have to think about human nature and the way we are used to, like, the way we've evolved over, you know, millennia to interact with others. Like, speaking feels so natural. Like, voice is— the models are finally getting good at it. It is getting faster. Voice, I'm like so bullish on voice.

Voice. What about you?

I think probably like Something that looks a lot like self-driving is probably going to be the thing that really becomes much more important and prominent in the next year. I think a big problem people have is empty input box problems. It doesn't really matter if you're using a general tool like ChatGPT or something more industry-specific. You have the same— it's like, okay, I have the product, now what? Well, now you've got to learn how to use it. These products are smart now. They're literally intelligent, so having them you know, kind of use themselves and sort of help you out and make that a really, really gentle on-ramp, I think it's probably going to be a very normal thing that we're going to look back on and be like, why didn't it always work that way?

Okay. We need the note-takers to be our accountability buddies. So we got voice, we got self-driving. I'm going to say hardware that replaces carrying around our laptop open. So that's hardware.

Hardware.

Yeah, like I want, I want like a little baby, baby bot over here, a little Codex in a box where I don't have to do— she's nodding— where I don't have to do this. That's my prediction. So whoever took notes, we're going to see which of us were right on what 2027 hot product of the year. Everybody, thank these lovely leaders from OpenAI for coming.

Thank you, Claire.

Thank you.