ChatGPT – The Super Assistant Era | BG2 Guest Interview

ChatGPT originally was entirely free, and the reason for that was that it was intended to be a demo and we were going to wind it down after a month. We then realized that the demo went viral and people loved the demo and it was actually a product. But we realized to be a product, you can't take the product down every time you're at capacity. So we shipped subscriptions simply because it could shape the demand. It was a way of gracefully turning users away when we had to turn away someone.

You know, you guys are at 900 million weekly active users now, and that growth has been incredible. The next billion users, where are they going to come from?

We've got about 10% of the world coming to us now. 90% left to go, right? There's so much more opportunity.

Well, Nick, so excited to have you here.

Thank you for having me, Apoorv.

You've had quite the journey. From Germany to the US for Brown.

That's true.

Most recently at Instacart, delivering groceries in 30 minutes to now delivering AGI to billions. I'm sure that was the plan all along.

Yeah, clearly total master plan.

Well, tell us about your journey. How did you get to OpenAI? I know it's a fun story. And your 3.5 years or so at OpenAI, how have they gone?

The only through line in how I made any sort of employment decision has been entirely people-based, so I don't claim any credit for joining OpenAI or predicting ChatGPT or anything like it. But I hit up someone who I admire a lot, who I got to know at Dropbox, Joanne, who worked here at the time. And I asked her to get off the DALL-E 2 waitlist, and she told me I had to interview if I wanted to get off the waitlist. I took the bait and got totally nerd sniped in the process. And here I am. There you go.

The DALL-E 2 waitlist will get you.

It's a great recruiting tool.

Nice.

We should do more waitlists, probably.

Yeah. Well, you know, the big supercycle we're in is ChatGPT. Now, I assume over a billion users on the monthly side, 900 million weekly active users recently reported. Up from zero 3.5 years ago. You could have, if I imagine what the dashboard of Nick Turley looks like, it could have users, it could have paying subscribers, it could have daily active users, it could have retention engagement. I mean, there's like 15 things, maybe all of them. What is your North Star? What are you optimizing for? What is Nick looking at in his daily dashboard?

It's funny, right? Because it's such a young product. It's been, to your point, 3.5 years. And this kind of question, it kind of changes as you evolve and you grow up and you ask yourself, what are we really building here? And to this day, I want to build a super assistant that can actually help people achieve their goals. And ultimately, the thing we care about is like, is our product doing that? Is it actually helping you do the thing that you're coming to the product to do? And it's so different for different people, right? Some people, are trying to get healthy, other people are trying to start a company, learn a new topic, do their taxes. There's all these different things that you might be doing. And the true measure of success is whether or not we're helping you do that. And obviously we look at WAU in particular because we want to know if you're coming back to the product. We look at retention. But we look at all kinds of stuff in aggregate because really there isn't this one single thing that you can optimize for.

If you were to allocate 100 units of points to these metrics, which metric can you distribute the 100 units across these metrics in order of importance for you right this second?

It's a good question. I care a lot about long-term retention and I would put all my points there because I'm really proud of the retention stats we have.

Huge.

But ultimately the sign of durable value is whether or not people are coming back in 3 months because that means you're really solving their problems. I think things like revenue, they follow from that versus trying to go on those things directly. And we've had a lot of success making very principled decisions on this stuff. One good example is GPT-4 used to be behind a paywall because we couldn't serve it to everyone. And then we had GPT-4, which was a total breakthrough in our ability to inference it. And so we just gave it away for free and that ended up being totally revenue positive. Retention positive because it just provided access to the tech. And I think when you make your decisions that way and you focus on the customer, you end up with a great product and revenue obviously follows too. Phenomenal. Yeah.

Well, it shows up in the numbers. I posted this chart yesterday on the data that we have from a third party. The retention curves for ChatGPT are smiling. Look at that. Just like that. And that is a rare, that is a very rare occurrence. As we know. And why do you think, if you were to give us a narrative on that smile curve, why do these smile curves exist? What are you seeing in ChatGPT that has people who have maybe churned off for a couple of weeks or months coming back, and why are they coming back?

Look, there isn't one single thing. The way that you build a retentive product is lots and lots of little things and really trying to make it better systematically. I will say that with AI and in particular ChatGPT, I found that it takes people some time to really understand all the parts of their life they can delegate. Right. And I think many users for that, it's a multi-month process for them to understand how can this thing help me and what are all the different ways that I can plug ChatGPT into my life. And but when I think about some of the breakthroughs and levers we've had, things like search and personalization, they have helped solve those user problems because search provides way more daily value to you. It used to be that ChatGPT was a pretty worky product. We'd see usage go down on the weekend. We'd usage go down during the summer months when a lot of people were off from work. And today we're mobile first. The vast majority of usage is mobile, and we see all these personal use cases. And I think search was a big investment that got us there. And personalization makes ChatGPT so much more relevant for you, right? Because it gets to know you over time, you get to know it. And those are two things that have materially moved the way that people come back to the product. But there's lots more to do. And as I mentioned, I'm not resting on our retention stats, even though we're obviously very proud.

Nice, nice, nice. And the other thing that I got wrong about ChatGPT was this is two and a half years ago. I was like, well, you know, let's look at who's going to win this consumer AI race. Typically these consumer markets are winner take most, winner take all. Look at search. Google has near 90% plus market share, $3.5, $4 trillion market cap. Mobile, same thing with Apple. Social, same thing with Meta. I was like, well, AI, Meta has all the distribution. Google's got all the distribution. They've got 3, 4 billion users. Well, it would be a flick of a switch for them to roll out their AI. But I was wrong. That's not what happened. ChatGPT turns out, you guys are at 900 million weekly active users now, and that growth has been incredible. Clearly, distribution was not enough, right? So the same question for distribution. What are the levers for us that have gotten us to the scale? Is it model quality? Is it product quality? Is it features? Is it the experience or product improvements like memory and personalization or search? Same question, like, what would you say drove historical growth and success?

We've got about 10% of the world coming to us now, 90% left to go, right? There's so much more opportunity to reach more people and introduce them to the way that AI can benefit them, right? But when I look backwards, and I only say that because the next billion users might be very different in terms of how you engage and reach and provide value. But when I look back, right, it's been roughly a sort of one-third, one-third, one-third between sort of classic friction removal type of work. Like one of the biggest moments when you look at pure impact was removing the authentication wall. And Sam will say, I told you so, because I think that was his feedback from like day one. It was like, you shouldn't have to log into ChatGPT. But it's like stuff like that that you do for any product and it does matter. Some things never change. Right? But then another third or so is what I would sort of core product investments. And they're really typically things that we've done together between research and product. So search and personalization are really good examples of that where we came together and we figured out not just UI/UX evolution, but also how to post-train these changes into the model. And it was really the moments when we came together. Another recent example is we have these writing blocks that render when you ask about queries where you're trying to write with the model and putting really good craft into those experiences really matters and our users love it. And then another third of the growth has been just model improvements, both step changes, like going from GPT-3.5 back then to GPT-4, than going from GPT-4 behind a paywall to 4.0 suddenly available to everyone, right? But a lot of it is also the iteration that isn't splashy, that doesn't warrant a named release. I'm really excited about the updates we just made with 5.3, 5.4, et cetera, because that is when we take a lot of user feedback and we methodically address it. And obviously that shows up in our retention as well. Sort of one-third, one-third, one-third between classic friction removal and access, core product investments, and then pure model improvements.

And so the question that I've really been waiting to ask you is how do we get the next billion? And talk about that a little bit. There's a lot of, it seems like at least from the outside fog of war, if I was a consumer today in the market to pick my super assistant, you would have a couple of great options. Claude out there, they're having some great traction last couple of weeks. Gemini, mega distribution, Uber distribution, and us, right? By the leading product today, at least in user numbers, the next billion users, where are they going to come from?

First of all, just to contextualize that goal, we care about 2 things at the end of the day. Obviously, reaching more people is really important. It's a direct manifestation of our mission to the world where the more people we can introduce to the benefits of AI, the better that is. But we're also really excited to go deeper. And that means taking the same billion users that find value in ChatGPT today and actually providing more meaningful value in the world, like actually helping them achieve their goals. Not just answering questions, right? So I'll talk about how we get to more scale, but I think it's important to remember that the way this technology is evolving is we're going to go beyond pure chatbots pretty fast.

Exciting.

I think on scale, it's shocked me how many people have found value in ChatGPT as it works today, because I don't think delegation is a natural skill for most, and ChatGPT is a pretty It's a power tool, right? You come to it, it doesn't tell you what it's for. You kind of have to discover it on your own and you have to use it and then you'll learn about this prompt that was really cool. And then maybe you're on Twitter and you really learn about another one or you're on Instagram and you learn another one. But the product, it's like a raw appliance. And I think for two, you know, on one thing we really need to nail as we reach the next set of users is a product that has a bit more of an affordance. Because I think for most people, they're very, very busy. And everyone, I think, in the world has intelligence-constrained problems, like problems that more intelligence could help with. But you need to frame that to people. And I still feel like we're a little bit too much like a computer terminal, and it needs to feel more like software, like an operating system of software. So that's one thing. Another thing that gets at the same constraint is beginning to be proactive. In a world where a lot of folks are too busy to delegate their problems to AI or don't quite know where to start, I think being able to help you proactively is really, really important as well. But I think all of these are product evolutions that we could make on top of the current tech. And the thing that gets me particularly excited is prioritizing our next-generation tech, our reasoning models, because the truth is when you look at reasoning, in ChatGPT today, it's relevant to a very small group of people. It's relevant for the people who are trying to get the most out of ChatGPT. But I fundamentally believe that reasoning, it's transformative. And if you can figure out how to productize reasoning in a way that works on people's behalf without them even knowing, and that looks very much like, you know, the model doing long horizon tasks on your behalf. It doesn't mean you encounter the concept. It just means it's benefiting you, right? So there's so much work to do and the product certainly has to evolve to be relevant for this kind of scale.

Yeah. One of the things that I've been hoping for a while, and Brad made a bet 2 years ago, when can ChatGPT help me take actions? When can ChatGPT help me be more proactive? And I think his bet expired end of last year. So he's, he's, we're very curious when is that coming out? I'll frame that for you because, you know, with, with, with, with search engines and Google, you know, two decades ago you could have got on the 10 blue links, you could have spent an hour getting the answer. You can now get the answer instantly with ChatGPT.

Mm-hmm.

Uh, and it feels like the next step is, is actions.

100%.

And it feels like the next step is, you know, with, you know, Pulse is a, is a great proactive product.

Mm-hmm.

Um, that, that, that feels, I have a post that runs weekly, but what I would really like is like, hey, Nick spoke about something and just find me, make sure I know that Nick spoke about this. Or like, hey, this XYZ thing happened that I cared about a lot. When is that going to get proactive? What is the modality going to look like?

Yeah, so there's two concepts. I think there's ChatGPT doing stuff rather than just answering. And then there's ChatGPT being proactive. And I think when you put them together, you start feeling like it feels like a super assistant because I think these things compound on the action-taking piece. Strictly speaking, ChatGPT can do stuff today. The action space is just very limited, right? It can search the web, which means it can use search tool or browser in the same way that a human would. It can make images, it can do all these things, right? But it clearly doesn't have the same action space that a human with a computer would have. And that is what we aim to build. Timing is everything on these bets, right? And I don't pretend to be great at timing either. When I look at past attempts that we've made, like the ChatGPT agent, for example, which kind of has capabilities like this, it was just slightly too early. The models weren't quite good enough to hit real escape velocity. And the problem is if you don't have escape velocity is that users don't learn to trust it. They don't even try. So when you look at a lot of things people were doing in the original version of ChatGPT agent, It was the things that happened to work, like migrating your file server into the cloud or something like that. Useful stuff, but very niche. Yeah. And as this stuff gets better, we just have to get it to a point where people try to use it for real meaningful problems in their life because then we can start hill climbing. And this has been the magic of ChatGPT where ChatGPT upon launch was good enough to get real attempts add use cases, even if they didn't initially work. ChatGPT was a pretty bad writer originally. It was a bad software engineer, but people tried and got enough value out of it that we could take those use cases and make them great. And I do think we're about to get to that point with general-purpose agents where it works well enough that you get at least partial credit. And because you're getting partial credit, you get really good tasks back. And then the magic begins because once you have a set of use cases that you can climb the hill on, we can make them awesome. So on TAS, I think we're close, but I think even people inside of OpenAI would have had a hard time predicting exactly when this gets good. We've been excited about it for a while. On proactivity, Pulse was a really great first step because what we wanted to build was a form factor where you're not prompting the model, like the model's prompting you. For the reasons that I described earlier, which is, you know, it's so hard for people to delegate and to figure out what their problems are. What if the AI understood your goals and the things you're interested in and just could start being proactive on your behalf? Pulse is limited in the value it can provide for you because it's not connected to your life and it can't take action. So it's producing information for you and people love that. You know, I love that. I've got mine running too. But I think the magic begins when you have actions and proactivity because then it can begin speculatively actually detecting, hey, you just landed where you were supposed to go. I'm gonna call a cab for you. Or if you're at work, it's like, hey, I proactively ran this analysis because I saw your metrics dropped. So I think these things really compound and we need to nail multiple of the building blocks to really achieve the transformation and the form factor that we hope for.

As you were answering those questions, I now have 15 more questions for you. So I hope you have 15 more minutes. But okay, one by one, we'll start with what you said on actions and tasks. Got it on timing, tough to say, but is there a shape or ordinality of tasks or agents that you think, hey, this is the kind of thing that's likely to come first whenever it does?

I mean, the thing that's already come first is the domain-specific agents, right? If you look at what's happening in in code, we're fully there. It's mind-bending, but we've got so many engineers who don't open their IDE ever. And for me, as someone who used to code and then unfortunately got very, very busy, it's brought me back in the game. So Codex and products like it is clearly a product that has escape velocity where people are absolutely using it for all kinds of agentic work. And if you just take what people are doing, and make it work even better, you kind of get all the way there. You know, I won't be surprised if you see this happen for other forms of sort of quantitative knowledge work, just because it happens to have the properties that code has. It's testable. You know if it worked or not. It's very RL-friendly. But the domain-specific ones already work. I think the thing everyone's working for is general purpose agents that just kind of work for anything. And that's why I think you need to win a consumer because it's very hard to train people into like, okay, it can work. Deep Research was a consumer product and it really was our first agentic thing out there. But I think what consumers want is I can just ask it anything and it'll do what needs to be done. Without any sort of retraining. And we'll get there, just a matter of time.

At least a psychological goal is flight bookings, hotel bookings, restaurant bookings, shopping.

All this stuff. There are so many consumer problems, and those are just the type of things that you would kick off, right? The minute you have productivity, there's things you don't even think of as agentic tasks. Like you're trying to get in shape. You don't think of that as a task you would delegate. Unless you have a trainer, in which case you do, but most people don't, right? But if the AI knew that, it could totally start working in the background for you over very long periods of time and getting you, you know, here's your fitness plan. Okay, I actually signed you up for this thing. You could imagine it being quite helpful if it's aligned with your long-term interests.

You're going to give Ozempic a run for the money.

We got to be careful what businesses we get into, but hopefully we can help.

That'll be great. Cannot wait. Cannot wait. The second thing you said was proactive users, and that might require us to go beyond chatbots. What's an example of a modality that might take ChatGPT beyond a chatbot?

So chat will always be close to my heart. It's the way we grew up, and it's an important modality to stay. I think it's less about chat and more about natural language to me, where it's the fact that you can express yourself to the machine in ways that are very natural to you, whether or not that's text, whether or not that's voice, whether or not that is structured UI that is rendered by the model. That is just very, very powerful and that's here to stay. But I think the thing—

SA Server.

That's right. For those that don't know, that's the name of our codebase, short for Super Assistant Server, because it's proof that this was always the vision and is always the vision. But the thing that'll change, I think, is that chat is a great way of expressing your intent. It's a good way of communicating with the machine, but it's not a great output where in many cases what you want back is an artifact. Like, here's your plan for your trip. Here is the analysis. Here is an outcome that I delivered for you. I just made you $5. This is what I want my AI doing for me, right? Yeah, totally. I mean, this is what people care about, right? And I think chat will always be there as the way that you sort of disambiguate your intent and you kick off the task. But I don't think it's necessarily the final deliverable. And I think that's the way in which we can evolve. So hopefully that's a very graceful transition because I'm very lucky and it's hard earned to have a billion people coming to you weekly for a thing that they love. But I think it's a great jumping off point because we have so much unsatisfied intent. From people where they're clearly trying to do something and ChatGPT is helpful enough, but it could be so much more helpful. And I think that's where we evolve.

Yeah. And you must be sitting on so much of this data where people are showing up to ChatGPT and attempting, as you said, 3 years ago, they were at least making the attempt.

Yeah.

So you might have at least the frequency histogram of like, here are all the things that people want to achieve with us.

We do. We have really awesome classifiers that run automatically. It's fully privacy-preserving, but gives us a sense of what use cases people have. And it's important, right? Because when you make a new model, we make a model update, you want to know what use cases just got better, what use cases got worse. And that's not always trivial to figure out unless you have really good analytics on the system. But so much of my learning is actually qualitative where I will just, you know, I have a habit of reaching out to a fairly random set of users to just figure out what they're doing. And I've never worked on a product where 3.5 years later you're still learning every time, because usually by that time you know what the use cases are that your product can deliver on. But our tech is so unusual in the fact that I keep learning about something crazy I didn't know was possible.

Wow, that's awesome. But basically a billion users, I suspect a small fraction of them are power users who are getting maybe thousands, maybe tens of thousands of value on their $200 subscription. The vast majority is middle of the pack. And then a few, call it casual users who are start using ChatGPT as search maybe, or teach me about AI or help me with my homework. What is your focus like maybe in those constituents, power users, casual users and early users, or however you frame it, what is our focus on for each of those three factions?

Yeah. Yeah. Well, first of all, I feel accountable to our entire user base. In fact, our non-users too, because products like ChatGPT can have real externalities on all humans. But when I think about sort of the way we build, it's really useful to imagine the extremes. One extreme being a user who doesn't care about AI at all, who has a busy life, and needs to be convinced of the value that we can provide because that forces you to really nail the interface and to expose the capabilities that are hidden in the model in a way that people can actually grok. And then the other useful extreme is our power user base, because power users are the users who teach us what's possible. It's actually impossible for us to do all the product discovery on our own. Simply because of how empirical this technology is and how much you actually learn post-launch. So building for each of those extremes can be valuable, but our user base is incredibly diverse and people have so many different use cases. And this is why I like to look at all kinds of different segmentations, not just frequency, but also what use cases are you coming to us for? But definitely huge variety in the ChatGPT user base.

Yeah.

I look up to macOS, for example, as an example where it really works for people who don't understand technology at all. It's entirely magical. But if you are a power user, you've got Terminal, you've got Settings, you can configure almost anything in macOS. And it's really beautifully done where the complexity is progressively disclosed. So you can interact with it and love the simplicity of it all, but you can also got all the knobs and And developers love it, right? And so I think this is kind of the inspiration for how we want to be in ChatGPT. That doesn't mean we always live up to it, but it means that building for power users is extremely important. And that's not just a property that I think is sort of aesthetically exciting. It's also really important in AI because it's the power users who show you what's possible. They are actually doing the product discovery because it would be impossible for us with such an empirical tech to do all the product discovery on our own. So the type of user who subscribes to ChatGPT Pro, who used Codex before it quite worked, who is now the strongest advocate of tools like tokens and teaching us what's possible, that is an incredibly valuable member of the community. And it might not show up in your weekly active users. It's just one number, right? But this is exactly why there isn't like a single North Star. And you really need to take these different segments very seriously. So I love building for power users. And you asked on token consumption, et cetera. It's so fascinating to see. There's people who get incredible value out of these products and watching what they do is very informative.

Okay, so you're very focused on the entire user base. Learn a lot from the power users. You know, the other thing I might say is the power users right now are getting a lot of value, almost too much value.

And a lot of— no such thing.

No such thing. The analog that is most common is the Uber and Lyft of the 2015 era, right? And, you know, it took a while, but I know you were thinking about it a lot. I know you guys are thinking about pricing quite a bit.

Yeah.

Maybe tell us a little bit about pricing. Right now pricing is pretty simple. Is there a path for folks who are getting a lot of great value to price that product differently and meet them where they are and the other way on the other side?

And pricing, there's no world in which pricing doesn't significantly evolve when the technology is changing this quickly, right? ChatGPT originally was entirely free and the reason for that was that it was intended to be a demo and we were gonna wind it down after a month. We then realized that the demo went viral and people loved the demo and it was actually a product. But we realized to be a product, you can't take the product down every time you're at capacity. So we shipped subscriptions simply because it could shape the demand. It was a way of gracefully turning users away when we had to turn away someone, and it felt like the fairest and most equitable way of doing so is saying, hey, if you really need this product, pay a subscription fee and you got it. Then we figured out how to make the product stable and we had the choice of do we keep the subscription thing or do we go back to free? And we realized we had consistently more tech that we couldn't scale, GPT-4 being the first example, because we had way too many free users to serve GPT-4 and we put it behind the Plus plan. And so, you know, the way we stumbled into subscriptions was, was sort of accidental by trying to just solve for the user. And it felt like the right way at the time to, to provide maximal access to our tech. Since then, we've had so many other breakthroughs, including test-time compute, where you can scale up intelligence kind of as much as you want, more or less. And it took us and the entire industry a little bit of time to turn that into product value. But we're here now. Our power users want to use more and more and more intelligence. And it's possible that in the current era, having unlimited plan is like having unlimited electricity plan. It just doesn't make sense because people may need a lot, a lot of electricity and they're getting a lot of value out of that. There's a reason you can't buy that, right? So obviously I want to be really thoughtful about the way that we evolve our plans and SKUs and subscriptions, but be incredibly I'd be surprised if it didn't change given the magnitude and profoundness of the technical breakthroughs that we've had and the product breakthroughs that follow.

Yeah, relatedly, so I imagine you're going to have something for the power users. What about the other side? How do we get the casual users into the wheel and still monetize them?

As mentioned, our business model will evolve and the North Star is access, right? We would like to pick a way of providing an offer. We want to provide an offering that maximizes the number of people who can access our most powerful tools. I think for the longest time that has been subscriptions. Subscriptions have the downside of the fact that in many markets people don't have credit cards or they don't use credit cards to subscribe to software. And we're interested in other ways that can maximize access of the tech. Our ads pilots are in that spirit. We really view it as a tool of bringing ChatGPT and our intelligence most broadly to anyone around the world. And it is an example of how we constantly need to evolve and figure out the best way to bring the demand in line with what we are able to offer.

Makes sense. Makes sense. The ads piece has been a tricky one because Sam has historically expressed reluctance about ads and you've got to maintain a lot of trust while delivering that. So I guess what changed?

I think we've talked about this several times in my history at OpenAI, and every time it came up, we said if we were to do ads, we'd have to be really thoughtful about the way we do it. So the first thing we did, starting end of last year, was to really engage the company on if we put ads in ChatGPT, how should we approach it? Which of the principles weigh? How do you preserve the things that are magical about ChatGPT while getting the benefits of ads, which is our ability to bring our most advanced tech to anyone regardless of their ability to pay. And I really love where we ended up on the principle side. On the experience side, we're very, very early. But on the principle side, I feel really proud because it's very important that the answer of ChatGPT be independent, as an example. Respecting user privacy is very important, and there's a lot to learn from the way that tech has evolved over the last few years. Or really last decade. And I like that the principles are out there before we've even really gotten started. Like, we're very early with our pilots. It's kind of interesting. I was obviously very anxiously and eagerly looking at our support inbounds and data. And the most common inquiry about ads is not, you know, how do I disable ads or turn off ads? But it's like, how do I run an ad? Because the entire ecosystem is really excited to be part of the story and to figure out a way to talk to ChatGPT users. So there's a lot more to come, but I'm very eager to get this right.

Yeah, I'm sure you guys will. Switching gears, Nick, something you and I have spoken about a little bit is distribution and partnerships. There's a couple of big partnerships last year. Apple, Reliance with Gemini. Those are two big user bases, right? A lot of India, a lot of the iOS users. Tell us a little bit about how you think about partnerships for ChatGPT to meet the user base and maybe specifically on those two as well.

Look, I think partnerships are a great way to bring two products together and to expose something like ChatGPT to people who might not otherwise have encountered it. The thing that I care about most when considering something like a partnership is what is the user experience and can we make it amazing? Because at the end of the day, when you look at what's going on in the market, you can get users to click on things, you can get them to tap any sort of product, especially if it looks like a product they recognize, et cetera. But if the experience isn't truly awesome, people will churn or they will at least not retain in the way that we've been lucky to retain them on ChatGPT. So for that reason, I'm super interested in paths like that, but it needs to be great. It needs to accrue to the user. We are very lucky to have a great brand and a recognizable product for many folks. And I want to make sure that anything we do is accretive to all that.

Nick, you are a master of trade-offs. You must be making a lot of trade-offs right now. Tell us about some of the trade-offs you're making. Tell us about a trade-off that you might be making that people don't appreciate from the outside.

There are a lot of trade-offs indeed, and for different reasons, right? The one I encounter a lot is trading off delivering on the people, on the use cases that exist in the product today and making them better versus productizing step-change technology that's going to generate a whole nother set of use cases. Because when you think about how ChatGPT came to be, it was a totally open-ended product. It was basically a user experience around a technical breakthrough. And we couldn't have told you all the ways that people find it valuable, but putting it out there was really important because it allowed us to discover and the world to discover what you can do. And then post-ChatGPT, we can obviously very systematically go and improve on the things that people actually want to use it for. And when you're at a company in this moment where you both have such amazing traction with what exists today and the most mind-bending breakthroughs on the research side, the balance you have to strike is making the core product you have better today with all the things that matter, latency, reliability, making the use cases really great that people come to with providing access to the step change. And we try to get the balance right, but we're a small team and we don't always get it right. And for that reason, that's one of the most difficult trade-offs that I have to deal with.

Nick, I imagine one of the hardest trade-offs you guys make here is those GPUs that are melting between ChatGPT, between Codex research. How do you guys allocate the GPUs?

That is a very good question, and I'll let you know when I figure it out. Just kidding, we've gotten a lot better at this. I really hope, by the way, to be at a point one day, and I've yet to reach that point, where we don't have to face this trade-off because it's really painful to have real user demand for products that you can't serve. If you only ever worked in software, that's an entirely unusual dynamic, right? Where you just, or you were limited by this zero-sum resource out there. The marketplaces have it, but I think pure software doesn't really have the dynamic, right?

Yeah.

So one thing we try to do, obviously, we prioritize our existing users first. We want to provide a fast, reliable product, and that is critical and table stakes. Then when you look at new capabilities, the sort of naive business school thing to do would be to probably look at revenue, incremental revenue per GPU or something like that. But this is where it's more an art than a science because we often have new breakthrough capabilities that are entirely zero to one. Deep research was one of those. We couldn't have told you, is there going to be demand for a consumer demand for a research product? But if you don't productize it to find out, you will never know. So this is where we have to be a little bit thoughtful on how we balance things that are no-brainers that people are really going to love with things that are brand new ideas. Then obviously on the research side, there's a reason that Mark has the job he has because a big part of his job is figuring out what research to fund. And obviously GPU is a big part of that. So very nuanced topic that we're continuously getting better at. But for me, the priority is always on our users.

Yeah. The other takeaway that I had is you don't have line of sight to a time when you won't have that problem.

It's been so fascinating because we obviously have been incredibly lucky to encounter more and more users who want to use our technology. But then the value that we're able to provide for each user is going up as well. And GPU consumption correlates pretty well with that value. And when you just look at token consumption per user, especially in the enterprise too, which is a massive opportunity, you see a lot of very GPU-hungry workflows. And yes, demand keeps going up even as prices go down.

This is a fascinating insight. People used to think that humans were, you can't kind of make more humans. Well, it takes 9 months and then 19 years. But you're saying, well, that's actually a more less finite resource than GPUs.

Yeah, I mean, on the human side, you can hire more humans, and obviously we've been busy doing that and bringing the best talent across functions to OpenAI. In the world with agents, you can also get more leverage per human. You can make your humans very effective at their job to do more. But GPUs are zero-sum, and if you don't have more GPUs, you really have to figure out how do you make very, very hard trades and I hate making hard trades for our users. Hence the desire to have more GPUs. But it's useful to start with the most zero-sum trade-off when you do your planning. So I think starting working backwards from GPUs is usually a pretty good idea.

Yeah, you know, one of the— we have all these external data sources for charts of users and usage and activity and retention and all those things. What we don't have is tokens per user over time. And I bet that chart is like a sweet line going this way.

I think internal is pretty good. Our internal employees is a pretty good indicator for what's about to happen. And yes, the charts are mind-boggling.

Yeah, fascinating. Okay, a couple of quick ones on the present before we go into the landscape, which is shopping. We just moved into a new house. Took some photos and we were hoping that all our furniture would magically appear that ChatGPT helped us paint. But, you know, a lot of recent updates on ChatGPT shopping. Tell us about it. What are you thinking on shopping as ChatGPT as a shopping assistant?

Shopping is one of those use cases that exist organically in ChatGPT today, and they work. You can ask ChatGPT about any purchase you might be planning and get pretty excellent advice. But it's also one of those cases where the experience that exists in chat today, it's not the perfect experience that you would want because shopping is very visual, for example. So you're going to want to actually see products and images and be able to compare and contrast, not just read walls of text. People care about the sources of, you know, where can I learn more about a given product, etc. And so there's a lot of work to do to make this discovery really, really good and allowing people to use ChatGPT as an assistant to find the right product to buy. And that's where our focus lies, is making that really great and making that really great in a way that works for our retail partners as well. Because as I mentioned earlier, there's huge appetite from the ecosystem to be part of the ChatGPT journey. And nailing the discovery piece has been the most promising focus area to date.

Nick, on ChatGPT, you must see a breadth of information. You must see a breadth of use cases that people are doing with ChatGPT. And tell us something about what does the world underestimate about ChatGPT that you have maybe been surprised by, or listener might be surprised by?

There's been a real change in the way that people think of ChatGPT over the last year or so, where it's increasingly like a true thought partner to people. It's not just a thing that, you know, answers your question, but it's a thing that you can— it's a sparring partner that you can actually think things through with. And that shows up in all kinds of demands ranging from life advice, where, you know, got a relationship problem, you can actually get a lot of value out of ChatGPT just helping you think through how to handle it and how to talk to your partner about it, all the way to a work setting where you're working on an analysis or you're trying to figure out how to frame something or you're trying to build something. And ChatGPT really shows up as a second brain of sorts. I think that's qualitatively different in terms of how the mental model it occupies with people. And you see in the usage patterns and the use cases that exist. And I think the more we nail things like proactivity, which we talked about earlier in tasks, et cetera, I think the more it's going to feel like a teammate in the workplace and like a super assistant at home. And I think that's going to meaningfully change the use cases that people come for.

Yeah, I've been the most high stakes thing I do with ChatGPT. Is we have a new baby, and when the baby's crying at 3 in the morning, ChatGPT, what's going on?

First of all, congrats. Second of all, I've heard this from all parents in my life, that ChatGPT has become indispensable as a thought partner. And it makes sense, right? If you have a really specific scenario or you think it's a specific scenario to you, ChatGPT really comes through and can help you build confidence. And I think that's such an empowering thing, right? Like, and I imagine new parents aren't always the most confident about what is the right thing to do. And if ChatGPT can make you feel like you have agency and control and, you know, I think that's really valuable.

Yeah, it's huge. Well, thank you for making ChatGPT. It's literally getting me an extra hour of sleep every day.

It took a village, but that is a great metric. You know, that should be the North Star metric is like incremental hours of sleep. That's a great one.

Incremental hours of sleep, incremental hours of joy.

There you go. I mean, you joke, but like we talk about this a lot and because spiritually that is pretty close to what we hope we can do, right? Is help you reach whatever you consider self-actualization. Yeah. Whether or not that's sleep or joy or any other goal you might have.

Yeah, yeah, yeah. Well, thank you to the village. We're gonna switch gears and talk about the landscape. Sure, there's a lot going on on the field. You know, how would you frame ChatGPT's differentiation to people out there? There's a lot of different products out there.

Look, it's the best time in history to be a consumer of technology. It is indeed, because you got options and the competition is intense. And I think that's beautiful. And it's actually good for us too, because if you were to pre-mortem why a company like OpenAI does not achieve its mission, it's probably focus because of the sheer number of opportunities that become possible when you approach AGI, right? And having competition and options out there, I think it forces us to focus on our customers too and on things that really matter, which aren't always the most flashy things, right? Sometimes it's latency, reliability, the quality of the user experience. So I think it's a really good thing. I think the biggest differentiation of ChatGPT is the team behind it because we're not static, right? Anything we build will get copied sometimes in ways that are high craft, sometimes in ways that are sort of checkboxes. And it's really important to us that we evolve the category and build the super system that we've always imagined. And I think the reason that I have confidence that that's possible at a speed that outpaces the dynamic of being copied is that we have an amazing team and that we have an amazing team across research and engineering and design and all the different functions that it takes to make something amazing. And I think our unique ability has been to bring those functions together to build something that is sort of at the intersection of useful and possible. Right in that moment. So my best answer for you is we keep pushing forward and we hope to be expanding what people think of this product as.

Last winter we had obviously what was called Code Red. Google had a great model. There was a lot of talk about it. Marc Benioff switching very vocally to Gemini and us delaying ads and health agents and shopping. Basically hit pause on everything, making ChatGPT better. Talk to us about that moment, both about what led to that and what was happening in that moment.

Yeah, so first off, code reds are a tool we use to create focus. And as you can imagine, when you're in a place like OpenAI, and this is what makes us special too, special to work here, is there are so many different things going on. It's a research lab. We are pursuing many different ideas, right? And there's been these moments where we've wanted the company to come together to solve a problem across boundaries, no matter what your project might have been. And last year we had one of those moments where we felt like we need to show up for our users. We need to focus the things, focus on the basics, like reliability, performance, the way that talking to the model feels, making personalization really great. All these elements that our users care about. And I loved it because it was really an opportunity to work with a bunch of folks who I don't normally get to work with on making the product great. And we just exited the code red, which we knew we would with the launch of 5.3, which is a great model for the everyday user. It's great to talk to. And 5.4, which is workhorse if you're trying to do real knowledge work. And undoubtedly we're going to continue to use the tool of a Code Red whenever we want to create focus. But I'm excited because I think ChatGPT is in a great spot.

Yeah. So Code Red is over now.

That's correct.

It's not the new normal.

It's not the new normal. We want it to be a special thing, but it is a tool I suspect we will continue to use.

That's great. That's great. And maybe tangibly, if you were to point at how did Code Red change ChatGPT or maybe the ops or how the team operates?

The thing I try to get foster with the team is focus. So we are certainly more focused than we were 6 months ago on the things we really want to nail. And some of those things are very behind the scenes, like latency, reliability, those kind of things. And some of those things are very concerted efforts, like involving ChatGPT into the Super Assistant. So focus is the main lasting artifact. And as you imagine, it's hard to stay focused sometimes when there's so much going on in the space. But that's the hard job. And you asked me about trade-offs earlier. Getting the team to focus on the things that really matter to users is certainly one of them. That's always worth it.

Yeah. In the back of my mind as I ask you that question is all the other founders that are in the arena right now. And just a reminder that, hey, Code Red is a tool for you. Wartime at Palantir, as we used to call it, is a tool.

Yeah, I think every company does it differently in terms of how you get stuff done. But I think it's really valuable to have terminology that means something, that signals to people it's okay to drop your other stuff and it's okay to focus on this thing together, even if that wasn't your original job. Yeah. I think it works really well at a place like OpenAI, but I imagine startups would have an equivalent.

Yeah. You know, one of the things that caught everybody's imagination on our team was what Pieter was doing at OpenClaw. Incredibly potent to put all the tools together. Obviously, Pieter is a great builder. Congrats on bringing on Pieter to the team. Tell us a little bit about what Pieter is working on and when might the billions on ChatGPT have something to to see there?

Well, first of all, I'm very excited for Peter to be here. I was excited to have another German speaker in the house. Oh, I didn't know that. He's Austrian, I'm German. So we were exchanging guten morgens. But the Open Claw is so inspiring because it brought to life in many ways a vision that we'd had in different forms, admittedly, around this kind of AI that is fully embodied, that exists across different UIs, that can do stuff for you, that has state, that has an interaction pattern that feels a little bit more like talking to a human. Because Open Claw allows you to interact in a very, very natural way where you can send many texts back and forth and it's very curt. And so there's a lot of elements of Open Claw that I think were very clarifying to folks across the industry. But the best, I'm super excited to just learn from Peter and bring into the company and figure out what we can do together. So there's a lot more to come.

All right, so now on to the most fun section, rapid fire.

All right.

You ready?

Sure.

We'll start with my favorite game, which is long short. Pick an idea, a startup, a business, a product that you love, you think you're very bullish on.

If I was starting a company today, I'm really excited about these companies that are going into companies and getting extremely hands-on and doing effectively professional services with AI because we've saturated all the emails. and you need to get proximate to the problems. So, that's, it's, it's those companies that, that, um, I'm, I'm paying attention to.

Fascinating. So this is, you know, this is an example. This would be like, hey, you're going and either acquiring or, or, or going inside an operating firm that has scale and, and, and, and, and a humming engine.

Exactly.

And making that a more efficient engine.

Yeah. Or, or, or just like, you know, you, you're doing contracts for, for, for, for customers that have really hard problems and you're actually going in and committing to solving the problem. Because outcomes. Yeah, because there's a reason I think that we've made so much progress on math and coding, but not on many other domains, because those are domains we are approximate to. We as people who work in labs, there's all kinds of other domains that we are not as approximate to. And if you get approximate, I think you can build something transformative. And I think this is more important now precisely because the easy problems have been solved. The obvious problems have been solved by the models. Credit where credit is due. I think NotebookLM is awesome and differentiated and helps me learn new stuff. I think it's great.

It's so good. Yeah, it's so good.

I think this is the example of you can innovate and you can build something totally different.

It's awesome. Yeah, yeah, yeah. It's so good, particularly for, I found it for some more technical learning to be a very approachable way to learn.

Totally. And it's really cool. I feel like AI, an underrated capability of AI is to just transform things into a different medium. And I think that's so important for learning. We just launched these dynamic math blocks which allow you to visually understand math inside ChatGPT. Learning is obviously a big use case for us too. And I think just being able to transform things from text to visual, soon from visual to video and all these different media, is amazing because people have such different ways of processing information. And some people are like auditory learners, some people are visual, some people like reading. So I think that's really magical and a great, great angle to take.

Yeah. Amazing, amazing, amazing. You know, one of the things I think about a lot is education and education for kids now in school. The world's changing so fast. I'm not sure our education system is changing that fast.

Yeah.

What advice would you have for, for students who are in school now, um, you know, who might have to adapt faster than the system around them might adapt?

It's a really good question and something that I've thought a lot about, um, myself. And, you know, the, I think the most important perma skill in this era is, uh, Curiosity, I think, because if the machine can answer all your questions, you better have good questions. And the only way to have good questions, I think, is to pursue the things you were actually excited about from an early age and throughout your entire life. And I reflect on this because the only reason I'm here and working on this stuff is because I thought it was neat when I got nerd sniped in in the interview process, right? And it was like, this is so cool. And so no matter what you're doing, I think that's an important skill is to be curious and learn to stay curious. And I think I'm confident that if you foster that skill, you will know how to adapt to an evolving landscape of tools and AIs and jobs. So that would be my advice.

Yeah, curiosity is has always been the problem of scale. Our friend Bill Gurley wrote about it in his book Running Down a Dream.

But curious to check that out.

Yeah. What is a job that gets more valuable, not less, as AI gets better, as AGI arrives?

Well, I think maybe the easy answer is being an entrepreneur because it's the best time to build ever. In terms of being able to self-actualize your idea. But maybe one that is maybe non-obvious is I think writing actually is very important. And it's not because the AI can't write. AI will become amazing at writing just like any other domains, but because I think the skill of writing forces you to be very clear on what you have to say. And even though prompt engineering is obviously going to go away and has gone away to much extent, the idea of expressing what you want to a machine requires you to be a pretty good writer and a very precise writer. So I would say that that is a, you know, in any profession that involves very clear writing and therefore thinking, I think is well set up.

Yeah, 100%. Honestly, I mean, this is the whole thing about slop, right? There's just so much.

That's the other thing. I think there's going to be a permanent need for high-quality, trusted, authoritative content. And tools like ChatGPT can help you discover that content.

Yeah.

But I think the need for amazing content is also here to stay.

And final question, what has been your AGI, fieldy AGI moment? When did you feel it?

I've had so many, honestly, and it's definitely not stopped. A few weeks or so after I joined OpenAI, GPT-4 had finished training, and I remember trying it out, and it actually, it didn't impress me at all, nor anyone else that week, because it kind of didn't work. And it's because we hadn't figured out how to post-train it. And I think seeing it go from kind of, wait, is this really a thing? Or was GPT-3 kind of it? To, wow, actually this is an entire step change with what felt to me at the time, who didn't understand much about AI at all, as like just some tweaks or some, a little bit of final stretch work was profoundly humbling. 'Cause you can realize that you might, it might not look like we are close to really powerful, useful AI, but we probably are. And then the moment that really, there was two things that GPT-4 did that felt like AGI to me. One is it could do poetry, and I didn't think it was possible for any AI model to do poetry. Just kind of fundamental philosophically, it just didn't feel like in scope. And then the other one was it could produce code that actually worked and compiled. And then my next moment where I stared at the ceiling just in awe was when I realized GPT-4 could just simulate an entire computer terminal, like a full computer with commands, et cetera. And I'm like, wait, how would this be imbued in a language model? And there's been so many moments since then, honestly. Reasoning was a moment. One of the moments was when I think Mark and I were giving a demo of reasoning in front of the whole company. And this was a moment where we were still trying to kind of find use cases that were hard enough for the AI, for the reasoning to make a difference. We're way past that point, we know. But at the time, I think we were having to do a puzzle in front of everyone. And I think one of the moments that made me totally feel the AGI is like we were in the middle of the demo and everyone started laughing. I was like, wait, What is funny? And then I stared at the screen because we're showing this chain of thought as it was streaming out of the model. And the model swore and said like, oh damn it, may have to adjust because it realized it had made a mistake in the puzzle. And the fact that it did that, but in particular the fact that it did that in a way that was entirely emergent from the RL process, completely blew my mind and made me feel quite humble about what else these models might be able to do. So that was one of those moments. And then most recently, watching people use Codex, like watching people walk around with their computer open because they don't want the task to end. Watching people who have never coded in their life make stuff and bring ideas to life. That feels like an AGI moment. So honestly, it just, it's just accelerating for me and it doesn't wear off at all. And everyone has a different thing, obviously, but those were some of mine.

Yeah. You know, it's 10 years ago, there was a product called Kite. I don't know if you remember. It was for software engineers. It was like an AI coding product. That's when I felt the hunger for personal AI. And nothing happened for 10 years and then everything happened in the last 10 months.

The timing thing is really hard because it's actually quite possible to predict where things will end up, I think, in terms of the kind of product and form factors you're going to have. But to know when it happens, it's really hard for me to make statements on anything between sort of eventually and in 3 months because of all the ambiguity around.

Well, that's a tight enough window. Now in 3 months is a tight enough window.

3 months is pretty okay. Try to stick to the 3-month plan, more or less.

Yeah.

Though my team would probably tell me we don't, but I try. But yeah, anything in between 3 months and eventually is difficult. Yeah.

Yeah. Yeah. Well, thanks for doing it. You've got a lot going on. This was a total treat. We're so excited to see all the great products you release for us. If we can do anything to be of help, let us know.

Awesome. Thanks very much. Thanks for having me.

Of course, man. This was fun.

As a reminder to everybody, just our opinions, not investment advice.