OpenAI's Research Chief On The Soup Wars, Poker And The Next Models - EP 46 Mark Chen

On the recruitment wars. I mean, this got a lot of attention.

Yeah, clearly.

And it looked like Meta was quite aggressive. What exactly does this tit for tat look like? What state— what stage are we at?

Yeah, I mean, there is a pool of talent, right? And everyone kind of knows who they are. And, you know, I think many companies have realized that one of the key ingredients, not the only important ingredient, but one of the key ingredients to building a great AI lab is to get the best talent. And I think not a surprise that Meta has been aggressively employing this strategy. You know, we haven't sat back idly, and I actually want to tell this story from OpenAI's point of view. Okay. I think a lot has been made in the media of, oh, you know, there's this unidirectional flow of talent over to Meta. But the way that I've seen it is, you know, Meta, they've gone after a lot of people quite unsuccessfully. So just to give you context, right within my staff, within my direct reports, before they hired anyone from OpenAI, I think they went after half of my direct reports and they all declined. And of course, you know, if they have something like $10 billion of capital per year to deploy towards talent, they're going to get someone. So I actually feel like we've been fairly good about protecting our top talent. And, you know, it's been kind of interesting and fun to see it escalate over time. Um, you know, uh, some interesting stories here are— is Zuck actually went and, yeah, hand-delivered soup to people that he was trying to recruit from us.

Uh, yeah, it was just a just to show how far he would go.

Yeah, I think he hand-cooked the soup and it was shocking to me at the time, but over time I've kind of updated towards these things can be effective in their own way, right? And I've also delivered soup to people that we've been recruiting from Meta.

You're doing a soup counting.

I've thought of, if I had an offsite, the next offsite for my staff, I'm gonna take them to a cooking class. Okay. And yeah, I mean, it's just been, yeah, but I do think, there's something I've learned about recruiting.

Did you cook your soup?

Ah, me?

It's better if you get like Michelin star soup, you know what I mean?

Yeah, no, no, no. I think Dayho is very, very good and probably better than any soup I could cook. But yeah, I do think there is something I've learned about, you know, just how to go aggressively after top talent. And I think the thing I've been actually very inspired by is that at OpenAI, even among people who have left for Meta, I haven't heard anyone say AGI is going to be developed at Meta first. Everyone is very confident in the research program at OpenAI. And one thing that I have made very clear to my staff, to the whole research org is we don't counter dollar for dollar with Meta. And the multiples that below what Meta is offering that people are very happy to stay at OpenAI gives me so much conviction that, you know, people really believe in the upside and believe that we're going to do it.

And Alex Wang, he used to be one of the math comp— the— yeah, I'm sure you guys hung out.

Yeah, I mean, I have hung out with Alex a handful of times, but We don't too much anymore.

Yeah.

I mean, yeah.

Why did soup become the thing? It was just, it just—

I don't know. You know, it's, there's been soup, there's been flowers, there's been anything you can think of under the sun. But I don't know. I think, you know, life's an adventure. I play into the meme.

Yeah.

Yeah.

Is there any, any poker strategy to employ? Like, as you're thinking?

Well, again, I think it really goes back to what I've said about the media narrative. The game is not to retain every single person in the org. It's to trust in this pipeline that we have for developing talent and to understand who the key people we need to keep are and to keep those. And I think we've done a phenomenal job at that.

We have a special treat today. I'm excited. Mark Chen is here from OpenAI. He's the chief research officer. He's somebody I've gotten to know over the last couple of years. Thank you so much for coming.

Yeah, it's been great to know you for so long.

I feel like, you know, there's a handful of people in this world working on this very important project. And I mean, you're right at the top of it. So it's so cool to have a chance to chat.

Yeah, thanks for having me on.

It's my pleasure. And I mean, there's a bunch of things that I want to talk to you about because I've gotten to know you, like we said, over those last couple of years. I want to get— I want, I want people to know a bit more about your biography. And but I also know there's going to be AI enthusiasts who want us to go deep on a couple of things there. So we'll try to do everything. I wanted to start just by giving people a feel for your job, which in my head— when you— I mean, just correct me if I get any of this wrong, but you're, you're, you know, Sam has been— he's really into research, he's the boss, he's kind of at the top of the food chain. But then you and Jakob are working together to shape OpenAI's research direction, and, and then you're in this additional part of this role is actually deciding which compute goes where onto these projects. So you kind of have to chart where OpenAI is heading and then the mechanics of how you're going to get there. Yeah. And this always strikes me as a horrible job because I picture people doing everything in their power to get GPUs from users. It's true.

People are very creative in the ways that they try to make backroom deals to get the GPUs they need. But yeah, I mean, it is a big part of the job, right? Figuring out the priorities for the research org and also being accountable for execution. So really to that first point, there's this exercise that Jakob and I do every 1 to 2 months where we take stock of all the projects at OpenAI. And it's this big spreadsheet, about 300 projects, and we go and try to deeply understand each one as best as we can. And, um, really rank them. And I think for a company of 500 people, it's important for people to understand what the core priorities are and for those to be communicated clearly, both explicitly verbally and also through the way that we allocate compute. All right.

What do we do at Core Memory? We cover innovative, fast-moving, forward-thinking companies, which is why Core Memory is sponsored by Brex because Brex is the intelligent finance platform for many of these companies. 30,000 companies from startups to the world's largest corporations rely on Brex's technology for their finances. They've got smart corporate cards, high-yield business banking, and expense automation tools that are fantastic. I hate doing my expenses, and Brex's AI software runs right through those expenses, figure out where we're spending money, and take care of so much stuff for you so you don't have to waste your time on it yourself. Go to brex.com/corememory to learn more and just, you know, get with the program. Let's get going. Let's get out of this archaic finance software and move toward the future. CoreMemory and Brex. So you've got— when you're talking about the 500, this— these are the 500. This is the heart of the research team. Yeah, they're an organization now that's thousands.

Yeah, of people. Yeah.

Okay, so And then in that, when you're talking about this 300 projects, I, I imagine, I mean, obviously some of those are the giant frontier models and then some are probably experiments that people are working on. And so like, how do you possibly keep track of all that and then come to some sort of conclusion about what merits GPUs and what doesn't?

Absolutely. So I think, um, it is very important when doing this exercise to keep the core roadmap in focus. And one thing that differentiates OpenAI with other big labs out there is OpenAI has always had core exploratory research at its core. We are not in the business of replicating the results of other labs, of catching up to other labs in terms of benchmarks. That isn't really our bread and butter. We're always trying to figure out what that next paradigm is, and we're willing to invest the resources, to make sure that we find that. Right. And I think most people might be surprised at this, but more compute goes into that endeavor of doing exploration than it is to training the actual artifact.

It must be. It's still got to be. How do you stop yourself from being persuaded by someone? Because everybody's going to put— when I think about this, sometimes I picture when I was at The New York Times, you would have this page one meeting where everybody wants to be on page one.

Yeah.

Everybody thinks their story is the most important story. They're all doing their very best job to tell you why this thing is so important. Everybody in that room has worked weeks, months on, on whatever they're pitching. And so it feels like life and death. And that, yeah, I mean, it just, it's, it seems so difficult for me.

Yeah. No, it, it is also a difficult process and I think the hardest calls you have to make are, you know, this is a project that we just can't fund right now. But I also think that's good leadership. You need to clearly communicate that, hey, these are the priorities, this is what we're going to talk about, these are the types of results that we think move the research program. And, you know, there can be other things, but those have to be clearly number 2.

And when you, like, you were talking about not being reactive to your competitors. Yeah, when I was looking through my notes, I don't know if I could go to the line quick enough, but I mean, this, this was like a point of pride that I saw that you feel like, um, some of the other companies are, well, you know, you guys were in this position where you were ahead, um, and, and setting the bar for others. And so, so they were reactive, right? To what, to what you had coming out. Mm-hmm. We happen to be doing this interview a few days after Gemini 3 came out. Right. And you know, there is a degree to which your rivals at times. Like, yeah, I mean, there's this back and forth going on and I know the benchmarks are sort of controversial how valuable they are, but, you know, people go ahead on these things. So how do you also, as time has gone on, maintain that luxury or that intellectual position where you feel like we're just going to do what we're going to do?

Yeah, I, I think AI research today, the landscape is just much more competitive than it's ever been. And the important thing is to not get caught up in that competitive dynamic because you can always say, hey, you know, I'm going to ship an incremental update that puts me in front of my competitor for, you know, a couple of weeks or a couple of months. And I don't think that's the long-term sustainable way to do research because if you crack that next paradigm, that's just going to matter so much more, right? You're going to shape the evolution of it, you're going to understand all the side research directions around that sphere of ideas. And so when you think about our RL program as an example of this, we bet more than 2 years ago that we're really going to crack RL on language models. And this was a very unpopular bet at the time. Right now it seems obvious, but back then the environment was, hey, there's this pre-training machine that's working great. There's this post-training machine that's working great. Why invest in something else? And I think today everyone would tell you, thinking in language models, it's just a primitive you can't have, can't live without. And so we're really there to make these bold bets and to figure out how we can scale and build the algorithms to really scale to orders of magnitude more compute than we have today.

And I get that intellectually in my, yeah, It gets harder as you guys started as a, like, basically a pure research company. When you look at OpenAI today, I mean, you have product line, there's parts of OpenAI that look much more familiar to a mature Microsoft or a Google where you have product lines, you've got all these different things that you have to serve. Typically, I feel like you guys are still young enough, so maybe you don't have these exact pressures yet, but you know, as those companies go on, it always becomes, well, We're more focused on the things that are serving the bottom line than spending a ton of money on research. Always seems to get like dwindled down over time.

Yeah. And I think that's really one of the most special things about OpenAI. At its core, we're a pure AI research company. And I don't think you can say that of many other companies out there. And, you know, we were founded as a nonprofit and I joined during that era. And I think The spirit is build AGI, advance AGI research at all costs and do it in a safe way, of course. But yeah, I actually do think that's the best head fake to really creating value. If you focus and you win at the research, the value is easy to create. So I think there is a trap of getting too lost into like, oh, let's drive up the bottom line when in reality, if you do the best research, that part of the picture is very easy.

And you started in 2018.

In 2018. Yeah.

So you feel like that soul, that, that, that—

Yeah, that core culture and that core nucleus, it's really persisted. It's still there.

What does Elon say? What does he say? He says we shouldn't call any of you guys researchers. It's just engineering, right?

Yeah. No, I think— Yeah, we know it's, it's true because I feel like once you have this hierarchy, and you elevate, let's say, research science as a thing beyond engineering, you've completely already lost the game because when you're building a big model, so much is in the practice of optimizing all of those little percentages of how do you make your kernels that much faster? How do you make sure the numerics all work? And that's a deep engineering practice. And if you don't have that part of the picture, you can't scale to the number of GPUs we use today.

So because I think they're— well, okay, but there is like a mystique that surrounds a researcher versus an engineer, you know what I mean? So are you—

were you—

do you feel like, um, it is better to kind of stay level-headed on that? Is that— is that kind of what you're saying?

Or, or, um, well, I, I just feel like researchers, they come in so many different shapes, you know. Uh, some of our best researchers, they're They're the type that, you know, they come up with a billion ideas, right? And many of them are not good. But, you know, just when you're about to be like, ah, is this person really worth it? They come up with some phenomenal idea. Some of them are just so good at kind of executing on the clear path ahead. And so there's just so many different shapes of researchers. And I think it's hard to just lump it into one stereotypical type that works.

That makes sense. Okay, I won't, I won't belabor you with too many competitive, like, rival questions. It's just since Gemini 3 did come out, I did wonder.

Yeah.

What happens with you personally or the team when one of your rivals puts it, like, does everybody go and look and see what it can do? Is there like a, is there a prompt or a question that you throw, you often throw at these new models to see what they can do?

Yeah, so to speak to Gemini 3 specifically, it's a pretty good model. And I think one thing we do is try to build consensus. The benchmarks only tell you so much. And just looking purely at the benchmarks, we actually felt quite confident. We have models internally that perform at the level of Gemini 3, and we're pretty confident that we we'll release them soon and we can release successor models that are even better. But yeah, again, kind of the benchmarks only tell you so much. And I think everyone probes the models in their own way. There is this math problem I like to give the models. I think so far none of them has quite cracked it, even the thinking models. So yeah, I'll wait for that.

Is this like a secret math problem?

Oh, no, no, no. Well, if I announce it here, maybe it gets trained on. But yeah, I do think It's one of the nice puzzles of last year. It's this, the 42 problem. So you want to create this random number generator mod 42 and you have access to a bunch of primitives, which are random number generators modulo primes less than 42. You want to make as few calls on expectation to these sub-generators as possible. So it's a very cute puzzle, but the language models, they get pretty close to the optimal solution, but I haven't seen one quite crack Crack it.

Okay. This is— we're heading down a direction I want to ask you about. Yeah, absolutely. Just before we get there. So I know you're— I've seen you, you're very competitive. You've also told me.

Yes, I am.

I think I found a match.

I love competition.

I hate to fucking lose somewhere.

Yeah, I really hate losing. I hate losing.

Yeah. So I'm picturing, I'm just curious if this is at all right. I mean, if you know, if we know Gemini 3 or whatever is coming out on a Thursday, I mean, are you up at midnight throwing that problem at it or is it not quite that drastic?

No, I mean, I think it's in long arcs, right? And any endeavor, like, I'm kind of a person who has obsessions. I think any endeavor you have to play the long game. And we've actually been focusing on pre-training specifically, supercharging our pre-training efforts for the last half year. I think it's a result of some of those efforts together with Jakub focusing and building that muscle of pre-training at OpenAI, crafting a really superstar team around it, making sure that all of the important areas and aspects of pre-training are emphasized. That's what creates the artifacts today that feels like we can go head-to-head with Gemini 3 easily on pre-training.

Okay. I want to ask about the pre-training because I've been talking to all you guys about this a lot. But okay. But you're saying that you're less obsessed about lobbing problems at these new models just when they appear and more at this, this long journey. Absolutely.

Yeah.

Okay. Okay. The reason I want to talk about the puzzle that you're at, I mean, I, you know, I first met Jakob before OpenAI ever started when he was doing a coding competition. And I got, I got like super into coding competitions for a while. There's this guy Kennedy. I don't know if he's still famous, but he was like the Michael Jordan of these coding competitions. And so I went to watch one at Facebook. Used to— I don't know if they still do, but they had an annual Hacker Cup. Yeah, Hacker Cup. And, and that's where I saw Jakob for the first time. And then I know you, I think, did math competitions in high school. Yeah, like probably grade school through high school. And then you also— did you also do IOI?

So I got into coding really late in life.

Okay.

It was a roommate in college that convinced me to take my first coding class, and I had all the hubris of a mathematician at that time, whereas, like, you know, math is the purest and hardest science, and that's where, you know, you really prove your worth. I mean, I think I was probably too into the competition back then, but yeah, I mean, it became this super rewarding endeavor. And, and, you know, it started out as purely a way to keep in touch with my friends from college.

Um, you went to MIT?

Yeah, I went to MIT. Um, you know, I graduated and every weekend we would just log on and do these contests just to keep in touch with each other. And, um, you know, over time I found myself having a talent for it. You know, I, um, started competing fairly well and then writing problems for, for contests like the USA Coding Olympiad. Eventually started coaching that team. And yeah, it's been a great community where I've met people like Scott that, you know. Yeah.

Yeah. Okay. So you, so I think lots of people might be familiar with like math competitions cuz they probably see kids going through that. The IOI and, and these coding competitions are a little bit different. I mean, it's, I mean, you'll know it so much better, but when I saw them, I mean, it looks like a, it's almost like a word problem that's a puzzle and you are trying to kind of find the most efficient and correct way to solve that. And you're in this race against everybody. And everybody's like writing code on their computer. And then, and then some people try to get there really fast, but then their thing kind of doesn't solve the problem. Right. And then, you know, there's like this trade-off, right? Absolutely. Right. And so, so you, you actually were on the MIT team?

No, no, no.

Something I did after, after, after college. Okay. But today you are like the coach of the US national—

Yeah. One of the coaches.

One of the coaches. Okay. And was it last year or the year before? Like the US, like we hadn't won one in a long, long time. Yeah. Yeah. Yeah, didn't we?

Yeah, yeah, yeah. So, yeah, I mean, the team, I mean, it's, you know, you can never predict what the makeup of top talent looks like every year, but we had a very spiky team, I think, 2 years ago. Okay. And yeah, I believe they won the Olympiad.

Because I feel like usually it's like China or Russia or like Belarus and Poland. I mean, right? Yeah. And so this, the big competition takes place in a different country every year. What does it look like? How many people show up?

Yeah, yeah. So they, they take the top 4 students from every single country. It is as much of a competition as it is a social event. You know, this is a tight-knit community. They all do go on to do phenomenal things. And yeah, it's this intense 2-day contest where each day you get just 3 problems, 5 hours to solve them, and you can really feel the adrenaline and know, all the pressure in the room. Um, but it's also great fun. I think, um, people settle down and they make, you know, lifetime friends through it.

What do you— and like, as coach, I mean, you're so freaking busy, man. How— what do you— uh, how much time do you spend on this? What does that look like?

Honestly, um, the kids are so self-motivated. Sometimes it's really about just managing their performance and, and strategy. Um, I think, you know, you're going to have good days, you're going to have bad days, you're going to have good hours within the contest, bad hours, and You can't let that get into your head. There's a lot of similarities between managing contestants and managing researchers. It's like on a much longer timescale, but researchers have good months, bad months. You can't really let those strings of failures get into your head because that's just the nature of research. And I think a lot of it's morale management at a certain point. Yeah, I think one other interesting thing that contests have helped me realize lately is when you put the models and deploy them towards solving these contest problems, which they're quite good at these days.

Absolutely.

Yeah, I was going to ask you about that. They work in a very different way from humans. We typically think of these machines as they're very good at pattern recognition. You can take any problem, if it maps to a previous problem, it's probably going to be able to solve it. But what I've noticed is in some of the previous IOIs, there's this problem like messages, very ad hoc. I didn't think the models would solve it at all, but actually one of the easier problems for the AI. So yeah, I mean, this is giving me the sense that AIs plus humans in frontier research, it's going to do something amazing just because the AI has a different intuition for what's easy and what's not.

So, okay, is it vaguely, you know, when DeepMind did the whole AlphaGo thing. You know, there was that moment where it was doing things human— it was playing in ways humans hadn't played before. So kind of like vaguely similar to that, or—

I think so. I think so. Um, I think really with GPT-5 Pro, right, um, there's been an inflection point in frontier research. Um, and one of the best anecdotes I have for this is, you know, I think 3 days after the launch, I met up with a friend who was a physicist, and he had been playing around with the models, felt like, you know, they were cute but not super useful. And I challenged him with the Pro model, just try something ambitious. And, you know, he put in his latest paper, it thought for 30 minutes and just got it. And I would say that that reaction in that moment It was kind of like seeing Lee Sedol during that, you know, Move 37, Move 38. And I just think that is just going to keep happening more and more for frontier mathematics, for science, for biology, material science. The models have really gotten to that point.

I was going to ask you this question, which is not very original because I think we've been doing this ever since Big Blue and all the chess stuff. But yeah, just as somebody who had followed all these competitions, if I don't know, there's a sadness when you start seeing these models solving these things that were like the height of these achievement for these very unique human minds.

Well, yes and no. I mean, I was good at competitive programming. I was never at the absolute top. And maybe this is a way to get revenge. No, I'm just kidding. But I do think there's certainly a moment for myself. We tracked Coding Contest performance while we were developing reasoning models for a while. And at the start of the program, they were not super great at the level of any average competitor going into the contest. And yet over time, they just started creeping up and up in terms of capability. And you still remember that moment when you walk into the meeting and they have where your performance is and then the model is exceeding that. Um, man, that was also a shock to me. It's just like, wow, we've automated to this level of capability so fast. And of course, you know, Jakob was there still a bit smug, but within like 1 or 2 months it was also surpassing him. So, um, yeah, no, the models are at the frontier today, right? Um, it's so clear by even through the results we've done this summer at Coder, right? Top optimization competitive programmers in the world. I think it achieved second place there. And so really it's jumped from 100th place last year to top 5 this year. And like, do you think we'll still be doing these competitions in 10 years? I think so. I mean, they're just fun. I mean, certainly a bunch of people who use it to pad their resume or get a job drop off from doing it. But I think the people who've always excelled at it the most are people who just do it for the fun of it. Yeah. And I don't think that'll go away.

When I was doing this story, I mean, they were telling me that like if you're from Russia or I don't know which countries, that you basically get like an automatic free ride to any university that you want. I mean, I see the guys on the US team go to like Harvard and MIT, so they seem to be doing okay. But it doesn't seem like the US has a—

Yeah, I mean, don't you think it's gonna— Yeah, I mean, interviews, right? They're going to be kind of broken going forward. And everyone's seeing this a little bit. And, you know, even college exams or college homework, it's all broken at this point, right? And I do think we're going to need new ways of assessing and gauging, you know, who's performing well, who's learned the material.

Where somebody's actually at.

Yeah. Yeah. So, I mean, I've had this idea here where maybe for our interviews, we should just have candidates talk to ChatGPT. And it's a special kind of ChatGPT where the model's trying to gauge whether you know the material or whether you're at the capability level to work at OpenAI. And you have to have this conversation with it that convinces it deeply you belong at OpenAI. And of course, you can't be allowed to jailbreak it and we'd look at the transcript after, but maybe tests like this will more accurately reflect in the future whether you know.

So you don't do that yet, but you're thinking about—

Yeah, just creative ways to revamped the interviews.

Yeah, yeah. Well, I mean, Silicon Valley is famous for doing the, like, brain teasers during— yeah, yeah, yeah, yeah. Um, so you— I mean, we talked— you were very good at math growing up, and, and I think you— were you born on the East Coast?

Uh, yeah, born on the East Coast.

And then you lived on the West Coast?

Did you go to the West Coast?

And then you lived in Taiwan for, like, for, like, um, grade school to high school?

4 years. Yeah.

Okay. And your parents worked at Bell Labs.

Yep.

So you come from like engineering stock. Yeah. Uh, I mean, it's a really interesting background. Um, just because you kind of got like a flavor for all these innovation hubs and, and especially with your parents being at, at Bell Labs, I must think.

Yeah. Kind of fascinating.

Yeah.

I mean, yeah, I just grew up in a very scientific environment. You know, dinner table talk was puzzles and things like that. And, um, I also got kind of the more traditional, you know, Bell Labs East Coast experience. On the West Coast, my dad came to do a startup. So a little bit of that kind of new company, got exposed to that when I was young as well. And of course, the big jump to Taiwan, right? And I think it's a huge culture shock. You wear uniforms in a school, it has barbed wire around the school, right? And, and also getting exposed to kind of that level of rigor. I think it was just a number of really great experiences growing up.

So like the schools were harder.

Um, or, well, I would say it was just much more kind of, uh, you know, it's just, there's a little bit less flexibility and freedom in the school system. Yeah. But I think it also teaches you something.

Yeah.

Okay.

Since day one, the Core Memory Podcast has been supported by the fine people at E1 Ventures. They are a young and ambitious VC firm in Silicon Valley investing in young and ambitious companies and people. Thank you so much to E1 Ventures for all your support.

Yeah.

And you knew you wanted to come back to the US for college?

Absolutely. Yeah. Yeah. Okay.

And then, okay, so you're, you're at MIT. Um, you're kind of like in this interesting group. I guess MIT probably always has an interesting—

Oh man.

Yeah.

2012 was such a great year.

Yeah. Like who was— there's sort of like an all-star list.

Oh, I mean, it was a great year. Like, um, I don't know if you knew like Jacob Steinhardt, you know, he's doing Transluc now. Um, He and I used to do projects together in computer science class. There is Paul Christiano, who's a bunch of really phenomenal—

He worked at OpenAI for a while.

Yeah, yeah, yeah. A bunch of kind of big names in AI came from that year.

And then we were talking about the competitive coding, like Scott Wu, who's at Cognition. I mean, he's kind of like famous now as like a meme on X for his math abilities. But you just got— you got to know him through the coding community.

Yeah, through the coding community. Okay.

Yeah. Okay. And then now I see the competitive end of you guys. The output of this to me look like poker these days. I think I was— we were at an event which I think I have to— we have to keep secret or something like that, the specifics on this event, but I think I'm okay to talk about this part, which is like, I late at night, I'm walking by this table. There's you, Scott, I think Sean from Palantir, and then, and like a handful of other people in a, like a fairly intense, it looked intense to me, maybe it's just not for you guys, but like a fairly intense poker game. So you guys, this is where you've applied your, your math and competitive skills now.

Yeah. I mean, Poker is a really fun game and I've talked about my life in terms of a series of obsessions. Poker was definitely one of these obsessions in the past and I think the big revelation for me in poker was, you know, it's so much more a mathematical game than a game of reading people and bluffing. And I think the more you learn about poker, the more you update in that direction, right? I think that's, you know, I used to be a terrible bluffer, and when you know it's mathematically correct to bluff, then it's so easy, right? It's like you don't feel any nervousness around it. And yeah, it's just so interesting that you have a game that I think is perceived as so human, but the underlying mechanics and how to win are so deeply mathematical. And yeah, I kind of thought about this the other day where, you know, there's something about that in language modeling too, right? You have this deeply human process of generating language, but there's this mathematical machine that can really do it as well as we can.

I think about that part all the time as a writer, and that I did all this philosophy back in college with Wittgenstein and all these guys thinking about these things. Yeah. Well, how do you find like an edge? If you and Scott both strike me as like supernaturally good at math, I don't understand how one of you is outcalculating the other person?

Well, no, I mean, it is mostly a forum for us to just kind of just hang out and catch up with each other. And, you know, today we don't take it as serious. I think there is an element to taking something like poker very seriously that takes the fun out of it. And so, you know, my obsession with poker, I think, has ended more than a decade ago. Now it's just fun.

You're just saying this because I saw Scott win both days.

You might be right about that.

He was taking it. He He was taking it quite seriously. And so, like, coming out of college, I mean, you had in some sense— I was—

Oh, I beat him on the plane, though.

You beat him on the plane ride home?

Yes.

Okay. All right. All right. So you did— you did— was it just you versus him or it was like a group thing?

Again, it may be 3 or 4 people.

Okay. I feel like a lot of— I feel like there's 3. I don't think I'm overgeneralizing too far, especially among like if you cut back to the 2018, right? Sort of timeframe, um, as far as like people who were in AI at a high level. I mean, a lot had academic backgrounds, right? A lot were math prodigies or had gone on to sort of take their math background and get into robotics or something, physics like that. And then, and then there's this other bucket, which is people who had gone to Wall Street and done high-frequency trading and quants and things like that. And so that was the first path that you took, was you went straight from MIT to Wall Street.

Yeah. I mean, I don't wear that badge with too much pride. And to be honest, like, you know, it was a path that was fairly common for, you know, very quantitatively oriented kids at MIT. And I mean, it was certainly a very meritocratic system, right? You could apply intelligence and there's a very concrete reward function, the amount of kind of profit that you would make. But I think culturally it was hard for me. It was a place where when you discover something, your first instinct is to just keep it away from as many people as possible because your knowledge is what gives you your worth. And so it felt like you would have out of this outgrowth of just even internally at a company, like these competitive dynamics and people weren't very trusting of each other. I think it was also felt like such a closed ecosystem, right? I think we today don't feel too much like, you know, when someone in HFT finds a breakthrough that makes their algorithm a little bit faster, no one else feels it, right? And I over time just kind of felt like, you know, I woke up after 4 or 5 years, we were competing against the same exact set of players. Everyone was a little bit faster, but had the world really changed that much for it? And it felt like time to do something else, right? It just, a bunch of things lined up back then. You know, there's AlphaGo match, which I think was a huge inspiration for a lot of people at OpenAI.

And did you play Go?

I did not, but I think the sense in which the model was able to do something creative, I really wanted to understand what was going on behind that.

So you're watching that happen and had you been at all reading AI research papers and things like that?

To be honest, no. And then I saw that event. It was really inspiring and that's when I started doing my deep dives into AI. So one of my goals after seeing that was reproduce the DQN results. This is a network that was able to play a lot of Atari games effectively at a superhuman level. And going from there, that's how I got my start into AI.

And you were doing that on the side?

Yeah.

So you just work all day and then go back and try to—

Yeah.

Okay. I mean, it is weird. I remember I was interviewing George Hotz, it might have been roughly 2018, Maybe a little before that, you know, and he'd just done this thing building a self-driving car on his own in his garage. And then, you know, I mean, it's George, so he says large statements sometimes that may or may not be like exact or spot on or apply to other people. But he's like, he's like, AI is still so young. You can, you can basically learn the whole field if you read, I don't know what the number was, 10, 20, 30 research papers. I mean, it is fascinating to me that it's like old in many ways, stretching back decades, but this particular moment—

It's very shallow. I always give this advice to people who are intimidated by getting into AI. It's so shallow. Just spend 3 to 6 months picking some project, like maybe reproduce DQN, and you can get to the frontier very quickly. The last couple of years has added a little bit of depth, but it's not anything like theoretical math or physics.

Do you think— is this a field where— I asked Jakob this the other day. I don't know why I'm obsessed with this, but, you know, like in mathematics, you see, you see people tend to do their best work in their 20s or to have the big breakthrough. And then it's very hard as they get older to like have that same kind of moment. Like what you're saying, you know, are we, are we dependent on, on young people reading these papers and then having some insight, or is this something where it's more you can keep going throughout your whole career?

I think you can keep going. OpenAI itself does have a pretty young culture, but I don't think you need to be young to do good research. I think there is something about being young and having less priors about this is the way it's done. I think over time you may develop your own vision. Which is a good thing, right? But it also locks you into a frame of mind of like, oh, you know, this is how research is done. This is how good results come out. And I think younger researchers tend to have a little bit more plasticity around that concept. Yeah. Yeah.

So your career at OpenAI is funny because it seems like you walked in the door and had a very important, large position from the get-go. But when you first got there in 2018, it must have been what, like 50 people?

Oh no, it was much closer to 20.

Much closer to 20.

Yeah. And it really looked like two teams back then. I came in as a resident. So someone who clearly not a specialist, not a PhD. I think I was Ilya's only resident throughout his tenure at OpenAI. So I was very lucky in that regard, just kind of learning the way that he thinks high level about research.

And a resident in this case is usually the right-hand person.

Oh, so it's someone who comes in usually from another field who OpenAI wanted to invest in and train up in AI. Yeah. And so I think the first part of a residency is like a 6-month compressed PhD and then going from there to just getting into deeper and deeper research projects.

So you're kind of like talking to Ilya every day. Is he kind of shaping that?

Yeah, yeah, yeah. He was responsible for my projects, for my curriculum, for my learning. And And I would just go to him for, hey, what's this about? Why did people pursue this? Okay. Yeah.

And you, I mean, I think if you go on LinkedIn, I mean, it would say you were the head of Frontier Research as your first job at OpenAI.

Oh, no, no. I was in IC for 3-ish years. Yeah. Yeah. So I was doing independent research projects. I worked in generative modeling because that was really kind of where Ilya's focus was at the time. And then only after a while did I start managing teams.

I think because most— you're talking about generative. I mean, most people point to DALL·E as maybe the first big project that the public would mostly recognize. Is that fair?

Yeah. Yeah. So I think that also marked the transition between when I was an IC and a manager. So one of my own kind of big projects. And one I'm still pretty proud of today is ImageGPT, this proof of concept that even outside of language you could put things like images into a transformer and the model would just internalize very good representations and understand the content of images. And it's kind of like a proof of concept that you can do language modeling outside of pure text and get really, really great representations and scale them to be state of the art with other methods. That was I consider a precursor work to DALL·E, which I was on the opposite side of managing. And I think between those, another project I'm really proud of doing IC work on is Codex, where we set up a lot of the framework for evaluating coding models and also did a lot of in-depth study on how you can take language models and make them very good on code.

So what made you pick OpenAI? Because I could see it two ways in my head. One is, is big fish in a small pond. There's interesting people here. I remember the OpenAI of 2018 with 20 people. In my head, it was like, this is probably not going to work. You know, Google seems like they've got this locked up and this is like a pretty small group of people trying to take on something that appears to require many billions of dollars of capital and I mean, this was even before the scaling stuff. It was just like Google had invested so much in AI already, kind of like in a different form than we think. But, you know, you're already kind of doing translation on your phone and things like that. So was that like a hard decision for you or you just stumbled into the OpenAI gig so quickly that—

Well, yeah, I mean, I think there are two things, right? You need ambition of vision. That was certainly what OpenAI had at the time, but you also need the talent to back it up. And, you know, I did feel like OpenAI was one of the rare places where the ambition was very large, but the talent was also large enough to fulfill that gap. And, you know, I was lucky I knew people like Greg before from college. And yeah.

And so, yeah, Greg, was it— you overlapped at MIT?

I think we did some math contests together. Okay. Yeah.

Back in high school.

And Yeah, I shot him a message actually, and I was like, oh, you know, I don't know if I have the right skill set, but this sounds like a place that's doing great work.

So it still seems nuts to like just come at this, you know, out of nowhere. And now you're like leading research.

No, it's surreal to me too. It's surreal to me too. You know, even that transition from IC to manager, I was very hesitant about taking it. I didn't. Know if managing was a skill set that I would be good at. And I was really enjoying IC work. I think I was having a lot of fun doing it, excelling at it, building really great collaborations. But yeah, I mean, it's really been a wild ride.

Yeah. Yeah. Well, okay. On that point, I mean, you've always struck me as a very nice, like, level-headed guy. I have to say, you know, there's parts of OpenAI's history that are quite dramatic, soap opera-like, a little Game of Thrones-y, you know, power struggles. And like, to me, to be a manager in that— I will say now, like, I feel like things are a bit calmer than they were. But when you look backwards, it just seems like, I don't know, you're saying you have to learn these skills, but some of this feels like opposite. I don't know you that well, but some of it feels opposite to your personality to have to like deal with all of that.

Honestly, you know, I've been lucky at OpenAI. I genuinely say that. And in the sense that I've had managers that have really advocated for me. You know, they saw my talent and advocated for me. I think When I was in IC, Wojciech, he was like, oh man, you should bet on him for Codex. And then later on reporting to Bob, I've never asked for a promotion or uplevel and it's just organically happened. And everyone along the way has given me great advice. I think part of growing in management is just getting the reps. I don't think there's any better place to get the reps than at OpenAI. There's always challenges to solve. And yeah, I think kind of developing that confidence, I actually think management is something where it's really just about the experience. And, you know, there's, I would say, less so talent involved in it.

Yeah. I don't want to, like, embarrass you, and I don't know if this will or won't. I assume you probably don't want to get too much into the coup or the blip or whatever we want to call it.

Yeah, we can talk about anything.

Yeah. Yeah. Well, I just— I've interviewed so many people about this now, and I'm also going to say, some of my gems for my book. So, I won't gazump myself. But there's a couple moments in there where you help get the researchers aligned around the petition to bring Sam back. And then I think just either a day or two after that, there's kind of like this speech given, I think at Greg's house, maybe?

I think Chelsea's house.

Okay. Yeah. And Um, you know, both of those struck me as pretty profound moments in— especially for, um, I guess like standing up with— for what you believe in and rallying the troops. I mean, yeah, like, um, you know, in a moment of crisis. I don't know. So did you— those, um—

yeah, I mean, that did feel like a very pivotal moment for me. I think in the days following the blip, right, there was a lot of uncertainty. And, you know, myself, Nick, Barrett at the time, we felt this responsibility of, you know, the wolves are at the heels, right? Everyone's getting calls from all these competing labs being like, you should come work here instead. And I just set this goal of I will not lose a single person. And we didn't. And it was just every day opening up our houses, you know, people could come here, they could, you know, have a place where they let out their anxiety. And then also just helping them keep in touch with the leadership team, having a way for them to feel like they could make a difference. And I think, you know, over time, people really felt the spirit of, hey, we're all in this together. How do we make a difference? How do we signal to the world that we're all together?. And we had been kind of driving back and forth between a couple houses, and we had this idea of like, hey, you know, we need to show the world that we're all seriously aligned and we're going to work for Sam. And that's when the petition came together. And the idea, I think, got solidified at 2 AM. We got more than 90% of the whole research org signed I think by the morning. And it was just everyone calling their friends, being like, hey, are you in or are you not? And yeah, I think in the end, you know, it was very close to 100 people signing that petition.

Well, I mean, that must have put you in something of a tough spot, though, just because especially at the outset, it was kind of like Ilya and Sam were on opposite sides and Ilya is your mentor. And then I know Ilya kind of comes back. Yeah.

Yeah. Yeah.

I don't know. Was that Awkward.

No, no, it was hard. I mean, it's a low information environment, but fundamentally it's just— and yeah, I mean, I think at the moment, you know, you could very reasonably conclude, like, was— like, did Sam do anything here? You know, is there— but would Greg and Yakov, like, people of super high integrity quit over that? I just felt like, you know, there was some part of the story that was being misrepresented. Yeah. Yeah.

With the, you know, Jakob's been there for a very long time. Like, what should people know about Jakob that they don't?

It's interesting because he's a super funny guy. Yeah, he's hilarious. Oh my gosh. He has this, like, sarcastic humor. And yeah, it cracks me up so much, honestly. Like, yeah, that's one of my favorite things about OpenAI today. Just like the level of alignment I have with Jakob. I feel like we go into a meeting, we can just bounce off ideas and quickly get to alignment and then deliver the same message and operate on different parts of a big roadmap together. And yeah, it's just one of the big privileges I have working at OpenAI. Yeah, I mean, going to that, actually, that point about keeping people together, I still feel that way about OpenAI Research.

Were you guys still under attack?

Yeah, no, we are a family. Yeah, we're always under attack. Look, when any— and this is how I know we're in the lead, right? Any company starts, where do they try to recruit from? It's OpenAI. And they want the expertise, they want our vision, kind of our philosophy of the world. And we've made so many star researchers, right? I think OpenAI more than anywhere else has been a place that makes names in AI today. Um, and I still feel that same level of protectiveness. Like, you come after, I'm going to do anything in my power to make sure, you know, they're happy, they're eye in the eye, and they understand, you know, how their role fits into the roadmap.

I saw, yeah, I mean, this is something I battled with as I was doing the book or even just watching events unfold in real time. Like when I go back through the history, I mean, you've got, you've got Ilya in 2012 making sort of like a big breakthrough. And then, you know, you've got Noam at Google in 2017 doing Transformers, and then you've got Alec Radford.

Yeah.

You know, like sometimes the story is these individuals really pushing the field forward, and it feels like a field that's still so young that you can have, have this individual. And then it— and it seems like there's this group of— I don't know what the number is, but let's call it like 8 to 10 who seem to have an ability to do that repeatedly, and they're really shaping where this has all gone. And so when I started seeing, like, Jon Schulman leave or Alec leave, and then, you know, there's kind of— it's like, wow, okay, well, if you've lost a chunk of this all-star team, how do you— it seems like a kind of field where you sort of can't just replace that. And yet, you know, it was, it was kind of like after that that you guys pushed forward on reasoning and some of these other spots. Yeah. So I don't know. I've intellectually had trouble.

I do disagree with that as the overarching way to do good research today. I think there's certainly a lot of top-down steering. We bet on directions, but people— OpenAI has this beautiful culture of being bottom-up in a very deep way too, where some of the best ideas just organically emerge from sometimes the most surprising of places.. And I think really the great thing has been just like watching some of these bets unfold, take shape, get scaled, and reasoning being a core example of that.

Yeah. Okay. So in this, but like this idea that how star dependent are we? Because you still see Google spend an ungodly amount of money to bring Gnome back, you know what I mean? Yeah. And so this makes me think, okay, This is how this works.

Yeah, I mean, I think it's a mix, right? Like you have to invest in your pipeline because I'm very confident in our ability to create stars. But yeah, there's certainly very good people out there and everyone knows that they're good. I think if there's one thing that, you know, on the flip side I've learned from Meta is, you know, OpenAI can also go very aggressively after star talent. And, you know, there's this very aggressive recruiting approach that you know, I've taken a couple of pages from as well. Okay. But yeah, I think it's— we should always just be trying to assemble the best team.

Yeah.

In service of the mission that we want to accomplish.

It's funny because it is like a relatively small world and like all you guys hang out even though you're like rivals. And then, yeah, it must be weird because I know you're friends with different people on some level. Yeah. And then you're also trying to steal all their best.

I mean, yeah, it's a brutally competitive industry in all fronts, right? But again, that's what I love. I'm a deeply competitive person. I hate to lose. And yeah, on research, on recruiting, all of these fronts, I'll work very hard on them.

It reminds me, because I'm like a semiconductor— well, I'm a history nerd, but just the early semiconductor days were not that far off. I mean, you had, you had all these semiconductor startups come at once. They were all pushing the limits of physics and somebody would discover something at one. They go to the bar and have a— it's like people, they're engineers, they can't like stop from like sort of sharing knowledge with each other. But, and then they're also getting pulled, like, you know, it's hard. Each company is, is kind of quickly getting this breakthrough in one way or another.

Yeah. I mean, you raise an interesting point of, you know, there is going to be some base rate diffusion of ideas. And, um, I think there's two ways a company can respond to that. You can create these deep silos of like, hey, you know, we're going to protect information all of these ways. I don't think OpenAI operates that way, and we don't think that's the right way to operate. We just will outrun other people as fast as we can. And I love the culture of openness. People in research freely share ideas, and I think that's the way to make the fastest progress.

And how do you, Sam, and Jakob now work together? I think people sometimes, if you read the announcements and everything, you can tell that Sam is research-oriented over, over like day-to-day running of the company, you know what I mean? You can tell research is more of his, his passion and even just like in the titles and in the way it's been organized, especially recently. You and Jakob are so deep on this stuff and I know Sam is technical, but you guys are like in it all the time. Yeah. And then, you know, Sam, He's having conversations with everyone. Yeah. I'm just kinda like curious about this dynamic between the three of you and how, I mean, are you guys, I mean, I guess you're not always alignment on, in alignment on what is going to get the resources, but, um, yeah, I was, I was just curious about you guys' dynamic.

Yeah. Yeah. So I mean, um, it, it's a very tight cohort, you know, I talk to Sam and Yakim every day. Um, and you know, with Sam, he loves research. He loves just learning about research. He loves talking to researchers. I think in some ways he's very effective at getting a pulse on the research org. And I rely on him also to just, you know, are there any hidden latent problems here? Go and find them out, you know, surface them to me. Jakob and I—

personality or techno?

It could be just small things like, oh, you know, like just even the way the office is laid out, like makes it harder for this team and this team to collaborate., and the two of them need to collaborate to help unlock this breakthrough that we want. I mean, all these things are very, very important. And I think Jakob and I, we spend a lot of time figuring out how to design the org for success. I think pairing people with the right strengths together, also how to incentivize people to work on directions that we find are important. Yeah, that's a lot of the work that we do.

The— and Sam, what, he, um, like, is he reading papers? Is he chatting with you guys?

Is— yeah, yeah. I mean, I think he, he does his fair share of reading papers. He talks to researchers and just understands how, how they think about the world, the type of research that they're doing. Um, and of course he's responsible for a huge umbrella of things outside of that.

All right, I'm gonna ask some nerdy questions now. I'm gonna try to, um, I don't know if I can do our cash level, but I'm going to do my best. And, you know, I'll ask. I don't know how top secret some of this stuff is, but, but anyway, well, maybe you'll slip up and we'll just get it out. You know, in the meetings I have been on, and I don't think I'm revealing because we talked about it, but I think I'm safe here. But, you know, pre-training seems like this area where it seems that Um, my sense is you, you guys feel like you've figured something out you're excited about. You think this is really going to be like a major advance. It was also, I think, either a neglected spot or something of a sore spot, you know, previously things weren't maybe working exactly how you guys had expected or hoped. Um, you know, like, what can you tell us about what you figured out and, and, uh, you know, some sort of frame of reference on, on we've seeing these periodic big leaps forward.

Absolutely. So I think the way I would describe at a high level the last 2 years is we've put so much resourcing into reasoning, into understanding this primitive and making it work. And it really has worked. And I do think one byproduct of that is you lose a little bit of muscle on your other functions like pre-training and post-training. In the last 6 months, Jakub and I have done a lot of work to build that muscle back up. I think pre-training is really a muscle that you exercise. You need to make sure all the infra is fresh. You need to make sure people are working on optimization at the frontier, are working on numerics at the frontier. And I think you also have to make sure the mindshare is there. That's kind of one of the recent things I've been focusing a lot on, just kind of directing and shaping what people talk about at the company. And very much today that is pre-training. We think there's a lot of room in pre-training. A lot of people say scaling is dead. We don't think so at all. In some sense, all the focus on RL, I think it's a little bit of an alpha for us because we think there's so much room left in pre-training. And I think as a result of these efforts, we've been training much stronger models. And that also gives us a lot of confidence carrying into Gemini 3 and other releases coming this end of the year.

The way I picture it in my head sometimes is that you guys have been on this, you've just been running so fast. The whole field has been running so fast. And so we're at a moment where it's like, okay, we've gathered up this vast volume of information from the internet. We've thrown it onto this supercomputer. And, and that, you know, ChatGPT pops out and then we're just on this, this like this incredible race that's going on. And so like when I hear you guys, I'm just trying to think about this in like a, it's a level set maybe for people who don't follow this as closely. So you, you know, in that initial moment you just had so much data, you're throwing it at this machine, you try to shape that data a bit. Initially, and what now we're just learning more efficient ways to shape that. Just, it's not— yeah, yeah, not always clear on what the mistakes were.

So I do think, yeah, you touch on something that I've been thinking about a lot, right? When you think about pre-training, you're taking human-written data and you're teaching the model how to essentially emulate it, right? It understands human patterns of writing. And in some sense, that also bottlenecks and puts a ceiling on the capability that you're able to achieve. You can't really surpass what humans have written when you're imitating what humans have written. And so, you know, you work on things like RL. There, you know, you can really provide steer towards the hardest tasks that humans can come up with. Have the model basically think outside of the box, outside of what it's learned from imitating humans and achieve higher levels of capability. But there is this kind of interesting problem now of how do you go beyond what humans are able to do today? And I do find a serious measurement problem there too, even in the sense of, like, can humans judge superhuman performance in the sciences, right? How would we know that this superhuman mathematician is better than that superhuman mathematician? And we really do need to come up with better evaluations for what it means to make progress in this world. We've been lucky up to this point. There have been contests like the IMO, IOI, really just gauging who's the top 1 mathematician in the world. But when the model capabilities go beyond humans, there are no more tests, right?

Okay. You just made me think of a question going back to the IOI stuff. I mean, and sorry, we're gonna come back. I just, you just totally popped in my head. I mean, like, often I would see the kids who were amazing at those competitions, they would get hired somewhere like a Google or Facebook or something, but they weren't always like the, you know, the top executive or the top, the, the most famous engineer afterwards. And maybe it was like by choice, but I don't think Kennedy, who was like the Michael Jordan ended up working at any of these companies, and that totally could be by choice. I'm not trying to disparage him, but it's not clear to me, like, even— okay, so it's not clear to me that the human who excels at that is necessarily like the greatest engineer you're ever going to have. And so like, yeah, I mean, if an AI is particularly good at— what are we learning?

Yeah, that's a thing. Quite like about working in AI, it's, I think, more so than in standard engineering culture, it is a meritocracy in that I've tried this many times before and learned this lesson many times before, but it is hard to put in someone to lead a group who doesn't have the respect of the researchers that they're leading. And I think this is more so the case in research than anywhere else. You have to make very strong technical calls of this is the right path when there's a disagreement, this is the right kind of project. And if you make those calls wrong, you lose the respect of your researchers. So one of the fun things in working with AI and creating a strong AI org is all my bench is very deeply technical and it's fun to talk to them about the technical things.

Yeah. Okay. And then, okay, on this, on, on pre-training again.

Yeah.

For a second. Okay. You know, like to me in my head, it feels like Transformers helped kick off this massive, massive leap. I mean, reasoning to me feels very comparable, if not even sort of more amazing. I mean, are we, when I talk to you guys over the last few months, my, you know, and I can never tell if this is optimism, if you guys are just putting the best foot forward when I'm chatting to everyone. My sense when I talk to you, to Greg, to Jakub, you know, to Sam, is, is that you guys kind of feel like you've been putting in hard engineering work for like 3, 4, 5 years that hasn't fully manifested itself. And so then I can never tell how excited or not to be. I mean, like when you guys are hinting at some of the stuff you're seeing, do you feel like it is— you can already tell that it is a comparable leap forward?

Yeah.

In terms of these big epochal kind of things, I think so.

You know, I think when we launched GPT-5, you know, we, we talked a lot about synthetic data as well. You know, there are many other threads of this form that that we think are holding quite a bit of promise and that we're scaling up pretty aggressively right now. And I think it's always about maintaining that portfolio of bets, taking the ones that are providing more empirical promise and scaling and supporting them at an even greater degree.

It was like 2 weeks ago, Andrej Karpathy, who used to work at OpenAI, he went on Dwarkesh's podcast and seemed to deflate some giant portion of the AI industry by saying, I think he was saying what, that AGI was like 10 years, 10 years off. And then when I hear, and then I, I heard Dario talking about a week ago, I mean, he seemed to be holding on very much to like massive scientific, just his, his, um, what is he called? The, the nation of genius. He seemed to be holding still on the kind of like maybe a little slower, but like a 2-year timeline on that. Um, yeah.

Yeah.

When you heard what Andre said, what did you think?

Yeah. I mean, I think Twitter, they love this like cycle of, you know, it's so over, we're so back, and, you know, whatever plays into the narrative at the time, I think, you know, just becomes amplified.

Yeah, I'm trying to make it click here, but let's—

The way I think about it, yeah, I mean, it's like AGI, I mean, everyone defines their own point for AGI. I think even at OpenAI, you can't get everyone in the same room and be like, hey, this is my clear definition of AGI and it's consistent. And so I kind of think about it as something like, you know, you're in the Industrial Revolution, right? Do you consider having machines make textiles? Is that the Industrial Revolution or is it the steam engine? Everyone kind of has their different definition. And I think we're in the middle of this process of producing AGI. For me, I think the thing I index most on is are we producing novel scientific knowledge and are we advancing the scientific frontier? I feel since the summer there's been a tremendous phase shift on that front.

Okay, like from stuff that you're seeing in terms— the, the first things that are jumping to my head are all these startups that are in the biotech space that are showing, you know, one-shot antibodies and molecules. But I have no idea if that— well, like, what are you—

yeah, yeah. So I mean, I was so inspired by that encounter with the physicist that went back and thought, hey, well, we should just create OpenAI for Science. And the goal being, I think for the small set of scientists today who realize the potential of these models and feel like they want to lean in and accelerate, we should do the best that we can to accelerate them. And I know there are similar efforts at other companies aimed towards pushing the scientific frontier, but I think what we want to do and I would say a little bit of a framing in terms of how we differ from, let's say, Google's efforts to work on science is we want to allow everyone the ability to win the Nobel Prize for themselves. It's less so about us winning that at OpenAI, which would be nice, but we want to build the tooling and the framework so that all scientists out there feel that accelerative impact. And we think we can push the field collectively.

Well, the discoveries that you're saying you're excited about, I mean, are there any others specifically that you've—

Yeah. So I think if you want a huge list of these, you can go on Seb's Twitter account. So recently there's a GPT-5 paper on an open convex optimization problem that is actually—

Wait, whose Twitter account?

Sebastian Buchholz. Yeah. And it's very related to some of the core ML problems that we're solving. I know there was— I think people kind of dismiss these things as, oh, is it just fancy literature search or something like that? It's quite a bit more complicated than that. And, you know, there's some examples I could go into.

But am I honestly overwhelmed at the moment? Because, you know, I'm sort of a generalist, but I cover biotech a lot and it's like every 2 days, man, I'm walking in and it's, well, we're making an AI scientist. We one-shotted an antibody and And then so like part of me gets excited and, you know, at least a handful of these companies I know the people and they're real scientists.

And yeah, yeah, yeah.

There's so much of it that I'm like, either something amazing is happening or everyone— it's, it's like, it's, it's kind of too much for me to be able to discern where reality is.

Yeah. I mean, I wouldn't be surprised if it's happening in biology. Um, personally I have the most expertise in, you know, computer science and mathematics and you know, we, we do have the experts there that can confirm that these are discoveries being made. So that's the thing that gives me the most confidence. But I'm not surprised at all it's happening in biology.

But like, what you're saying is, is kind of different than the— I agree, the narrative changes every like 3 weeks, it seems like. But like, what you're saying is, is sort of different because the biggest knock, even before Andrej said that, it seemed to me from the You know what I was listening to? I was listening to like a politics podcast. Sagar, I think it's Breaking Points, is their podcast. You know, he's, he's pretty smart guy who's knowledgeable. But I mean, he's just been shitting on AI and the lack of progress. And this is all like make-believe and all in. And so, you know, if these discoveries are happening, I mean, I feel like the— yeah, I mean, aware of this.

Just to be clear, while setting up OpenAI for Science, we've talked to a lot of physicists, a lot of mathematicians, and actually most of the people we've talked to aren't that bullish on AI. I think they still believe, hey, this thing isn't something that can solve new theorems. There's no way it could do that. There must be something else going on. And that's why I feel like empowering the set of people who really do believe and lean into it, like those people are going to just, you know, outrun everyone else. And we want to build the tools and convince people like this is the right way to do scientific research.

Okay. And so, I mean, so like on that point, I mean, I grant you that everybody's definition of AGI is different, but you're like, at least what I'm hearing is, I mean, you, whatever you want to call it, you feel like in the next year or two we're just seeing dramatic things.

Yeah, I mean, it is a bit of a meme, right? It's like you ask someone, when is AGI? It's 2 years away, right? And I don't think we're in that world anymore. And it's like these results in math and science that are giving me this conviction. But at OpenAI, within the research org, we set 2 very concrete goals, right? Within a year, we want to change the nature of the way that we're doing research. And we want to be productively relying on AI interns in the research development process. And within 2.5 years, we want AI to be doing end-to-end research. And I think it's very different, right? Like today, you come up with an idea, you execute on it, you implement it, you debug it. It means within a year, we're quite confident we can get to a world where we control the outer loop. We come up with the ideas,, but the model is in charge of the implementation, the debugging.

Yeah. Okay. Are there, beyond pre-training, when I talk to you guys, sometimes I get the sense similar sort of thing. It's like we all have in our heads, at least people where I sit, that there's been this massive infrastructure build out, that the models seem to get better every time you 10x them, that there was a story for a while that as you guys were going from like 4 to 5, you weren't seeing the results you wanted, even though you were getting more compute. But then the more I talked to you guys, the more it sounds to me like you feel we haven't actually— that things were moving so fast back then that we haven't actually seen the moment where we made the leap to the 10x. Computer?

I don't know if I asked that question very eloquently, but yeah, I mean, I do have a thought to share here, which is when people ask me, do you guys really need all this compute? It's such a shocking question because day to day I'm dealing with so many compute requests. And really my frame of mind is if we had 3x the compute today, I could immediately utilize that very effectively. If we had 10x the compute today, probably within a small number of weeks, fully utilize that productively. And so I think the demand for compute is really there. I don't see any slowdown. And yeah, it almost baffles me when I hear people ask, do you guys really need more compute? Yeah, it doesn't make sense to me.

And you think we, in the broad strokes of the question I asked badly, along the lines of where you guys seem very optimistic about what you've cracked on pre-training, are you equally not just like this demand that people want more GPUs, but are you, are you, do you see pretty clearly that, that same thing scaling is about to kick things high?

Yeah, yeah, we absolutely want to keep scaling the models. And I think we have algorithmic breakthroughs that enable us to scale the models. And, you know, I think there's a lot impressive about Gemini 3. One thing that kind of reading into the details, that I've noticed is, you know, when you look at stuff like their SUI bench numbers, there's still a big thing around data efficiency that they haven't cracked, right? They haven't made that much movement on it. And I think we have very strong algorithms there.

Yeah. Well, and there was this leaked memo from, I mean, Sam was sounding quite somber about Gemini 3, man, in this memo. I'm trying to find the quote. Did you, well, you obviously, I'm sure you got the memo. It's like, it, it, it seemed like a bit of a moment.

Um, yeah, well, I do think part of Sam's job is to inject urgency and pace, and that's also part of my job as well. Um, I think it is important for us to be laser-focused on, on scaling, and I do think, you know, Gemini 3 is exactly like the right kind of bet that that Google should be pursuing. At the same time, I would calibrate that by saying, you know, a large part of our jobs is to inject as much urgency into the org as possible.

Yeah.

And it is a good model. I think we have a response and I think we can execute even faster to the follow-up.

How much do you get involved with things like And I'm sure you're going to tell me exactly what it looks like with Johnny Ive's device. Cool.

Yeah.

Yeah. Like, is that an area that research plays in?

Yeah, it is. And actually, I was just having dinner yesterday.

You can describe it to me if you want. Yeah, absolutely.

So it looks like this. Well, yesterday I was just having dinner with Johnny, with some researchers as well, our head of pre-training and also post-training. Really the way I think about ChatGPT in the future, right? Today when you look at how you interact with ChatGPT, it feels very dumb to me. It doesn't feel very thinking native, right? And you go to it with a prompt, right? You get a response and then it's doing no productive work for you until you give it the next prompt. And if you give it a similar prompt, you know, it's going to think for the same amount of time. It hasn't gotten smarter because you added ask the first prompt. And I think the future is going to be a world where memory is going to be a much improved feature. Every time you go to ChatGPT, it learns something deep about you. It reflects about why you would ask this question, related questions, anything. And then the next time you go to it, it's going to be that much smarter. And I think it really begs the question of how do you design a device that has this as the dominating thesis. And yeah, I thought that's been a very productive experience.

Do you have one?

Do I have one? I may or may not.

What I think about when I think about you guys talking to Johnny is that like at Apple, you have this company that was centered around hardware. It's something that Steve Jobs obsessed about all the time. It's like, you know, it's a craft. It's like an art form. Um, whether it's you, Sam, Greg, Jakob, whomever, as far as I'm aware, none of you guys have really done a hardware product before. Um, Sam seems to take design very seriously. I could tell from his, the buildings in his house and things like that. But you know, there's no sort of track record to speak of, of like, I always thought of Steve Jobs as having like taste, you know? And then, and I've, I've had a couple bosses over the years, like Josh Tierringale, he used to run Businessweek. He kind of, he just always struck me as this guy. He just had taste, you know, whether it was the way something looked, the way a story should be. There was like this innate, thing that was on this really high level. It strikes me that's kind of like what's required here. I guess that's why you have someone like Johnny on some level. Absolutely. But you have to have this like back and forth. How do we know that like any of you guys have taste and, and are, you know, can shape a hardware product?

Honestly, we don't need to have taste ourselves. And that, that is Johnny's job.

Yeah.

He's our discriminator on, on taste. And I think actually one thing that's been really nice is just realizing that the way they work in design and the way we work in research is there's some deep parallels there, right? There's like so much exploration and ideation and you explore a bunch of hypotheses, you take your time and then you create kind of the thing that you're happy with, the artifact at the end that you're happy about. And yeah, it's been really nice to kind of have them fold into the company and there's just a lot more direct communication about here's the capabilities that we're going to ship, and here's what the form factor looks like and how to gel them.

Okay. And I— this is like a crass way to put this, but, um, because I spend my life adoring and talking to these people, but, you know, sometimes I'm just like, man, I just don't know if a bunch of math nerds are the ones that you want making like, uh, the AI computer, you know. But I guess it is this blend that you're talking about.

Uh, yeah, I mean, honestly, um, You're right in that the people who are the best at building AI capabilities are slightly different from the people who have the best taste. And we do have teams built of people who have really great taste for model behavior. And I think there's a very different kind of philosophy and a very different kind of set of questions you need to keep asking yourself. One example of a good taste question, right? Like you can imagine this being like in the model behavior interviews, like what should ChatGPT's favorite number be?

What should its what number be?

ChatGPT's favorite number.

Oh, okay. Okay.

I'm curious what you would ask.

You would answer what I think its favorite number should be.

Yeah.

Well, I have a stupid answer, which is that I went to Pomona College and 47 is this like number of lore there. So. Okay. Okay. Okay.

Yeah. I mean, that's a good answer.

Yeah. The— okay, I'm going to let you go in a second. You've been really generous. I appreciate it. Is there— well, I'm going to ask you a question that ChatGPT told me to ask you, which is, is, you know, it says like, if you look back in 5 years, are there any kind of like small, fragile, nascent ideas that you're seeing right now that you, you your instinct is telling you might be at the heart of a big breakthrough?

Yeah, there's a couple, I would say a handful of ideas. I can't go into too much detail on them, but yeah, I'm really, really excited to scale them up.

Are there any hints, any buckets of areas where these fall?

Yeah, I mean, I've been concentrating a lot on pre-training, so some pre-training adjacent. A small number of ideas in RL as well, and a small number of ideas of how to put it all together.

Yeah. Okay.

All right.

I tried. I tried. So you may or may not have a device. No, no hints.

Yeah. No hints.

Okay. Well, we covered tons of ground. I really appreciate it. Is there— I feel like I'm letting the nerds down a little bit as far as like the AI Obsessives. Yeah. Are there any technical— anything you see people like kind of getting wrong about you guys at the moment that you would set the record straight on?

Yeah, I mean, I think the most important thing is just— I think anyone at OpenAI in research would tell you that it is just a research-centric company. It's a pure AI bet at the core of the company. The ambition is to build AGI, it's to build it without distractions. And I think anything when it comes to building products, it all flows very easily from that. When it comes to what we want to do in research, we want to automate AI research. I think selfishly, we want to accelerate our own progress and then we want to automate scientific discovery. And of course we want to automate the ability to do economically useful work. And I think all these pillars are falling. And you see that kind of the big update in the last year has just been like in that second pillar of automating scientific research. It's happening. Yeah.

How old are you now?

34, about to turn 35.

About to turn 35. Okay. Are you able to have like a Social life, or are you?

No, honestly not. I think every day the last 2 weeks, you know, it's been work calls till 1, 2 AM. But I love doing it. It's just, you know, there's a lot of work to get done. There's a lot of people I want to recruit. There's a lot of steering that needs to be done. And like, why waste this golden moment? It's like if we're in the middle of something like an industrial revolution, you got to take as much advantage of it as possible.

Yeah, I hear stories about you sleeping at the office and—

Oh yeah, that was a fun one too. No, honestly, it's just, yeah, I think there are times in the company. I think that was right after Barrett and Mira left and went to found their own company. It just— the job demands it. And I think when I peel it all back and examine that deep emotion. It's just this protectiveness of, of the research.

That was after Mira left.

Yeah. Yeah. I spent, spent a month kind of sleeping in the office, and it's just like, I, I need to protect the research org. They feel like, you know, it feels like my baby.

Yeah. So you guys have gone through these waves. There's the coup. Everybody's trying to steal your people. I guess everybody's trying to steal your people all the time. But you have this inflection point. Mira leaves. Meta decides they're going to fire up this massive lab. Do you think— are we like, are we past it? Has everybody fired their, their shot at this point?

I have a staff meeting, I talk to my reports and I'm like, okay, well, here's the thing that I'm working on. And once I get back, once I'm done with this thread, I'm going to zoom out and there's no more fires. I've fully internalized at this point, the stakes are high enough for building AGI that there's always going to be something. And I think the important thing is just being able to understand what the important things are in the midst of all of all of these, these things going on.

Do you— like, months have passed since there was sort of the Deepseek moment or whatever. I guess it was like December 2024, I think. Yeah.

Yeah. Earlier, earlier this. Yeah. Yeah. Yeah.

Or January.

Yeah.

Yeah. I mean, is there anything now, you know, it felt like people lost their minds for a second, just like reflecting on it now and seeing what they've done since. Just, just like thoughts, I guess, on open source models and Chinese open source models.

Yeah, I mean, I think that was one of the first points in time when I just realized how important it is that we just stay true to our research format. I think when, when that came out, you know, it went viral, right? Like everyone was like, oh man, like, has OpenAI lost its way? Are these models catching up? And what's the response? What's the response? What's the response? And I think rightfully The thing that we did was we just doubled down on our own research program and I don't think it was the right, it was the wrong call at all. I haven't seen the DeepSeq follow-up model. I think they're a very strong lab, but fundamentally let's just keep focusing on innovating. I think DeepSeq was a great kind of replication of the ideas in our O series of models, but let's just focus on innovating.

Do you think 500 people is the— does that number grow as the company grows, or this is like the optimum number for kind of like big ideas you can chase at one time?

No, honestly, I feel like it can be done with even less. And again, as we get AI researchers, AI interns, there's a real question of how do you design your work around that. But I'm certainly a person who cares a lot about heavy talent density. I like to run a lot of experiments of this vein. For instance, in Q2 of this year, I thought, hey, I'm just not going to open up any headcount for anyone in research. And if you want to hire people, you got to figure out who's not on the boat. And I think these kind of exercises are quite important. You know, you don't want an org to diffuse into something that's not manageable and you want to keep the talent bar very high.

Okay. I promise this is the last question. Yeah, absolutely. Yeah. Sorry. I have to set you free. I remember being in a meeting and I think you and Jaka were kind of on the same page here, but I remember you for sure. Um, sort of like this idea of, of who gets attribution for a project. And you seem to be of the stance that like people are obsessing over that a bit too much. And, and clearly AI has its roots in academia where you are very proud when you have a paper and it's a big deal and, and attribution is a huge thing. Um, I think I'm remembering that meeting. Right. And you're—

I think I know. Yeah.

And, and so what, we've reached a new stage where that is less of a big deal or it's just you— this is a company and who did what is less important.

I actually really love this topic and I think over-fixation on credit is a very bad thing. Right. I think, you know, but on the other hand, I actually feel like it's important as a company for us to recognize credit both internally and externally. And a lot of companies have actually shied away from this. You know, we've moved away from publishing papers, credits lists, I think broadly throughout the industry. But Jakub and I ended up making the call that we are going to do it at OpenAI. And Of course, the counterargument is always like, man, you're like handing your top performers on a platter. Everyone else is going to be recruiting these guys aggressively. But I don't think that's important, right? Like we should just recognize the people who are doing great work. We should continue to be this pipeline for creating AI superstars. And yeah, honestly, it's important for us to make names for the people who are doing the best work at the company.

But you seem to also be saying people, the individual researchers, should maybe obsess about this less. Or am I, am I just totally misremembering this?

No, I, I think there was a sentiment in the room of, of that form. Actually, Jakob and I held more of a dissenting view on that.

Oh, okay. Okay.

Okay.

Okay. It's been a while. It appears in my notes.

Perfect.

Yeah, yeah, yeah.

But I think we got to give credit where it's due, even at the risk of everyone knowing who our top talent is.

Okay. Okay.

I will make an even stronger statement. I think OpenAI is the place where we allow for the most external credit per capita.

Okay.

By a large margin.

Okay. All right. Well, I'm going to check my notes. Well, now I've got more.

Absolutely.

Absolutely. Yeah. I just remember it being a topic of discussion and there were numerous opinions. So that's funny. Okay. I lied. Last question. So, you know, you got there in 2018. I mean, it was a research company, it was a nonprofit. The company started among the founders, you know, being this counterweight to Google and with sort of, you know, making sure AGI arrived safely was kind of the goal. You came at this from high-frequency trading and saw these interesting things happening, you know, like how much in your— I'm sure you're gonna say you want this to happen safely, I get that. But like, if you look at your career path, you're a smart, curious human who saw this interesting thing happening. It's not like a requirement that you like really give a shit philosophically about this or want to see, you know, a superintelligence. Yeah, I mean, but anyway, like, let's hear from you on on why are you doing this in the first place?

Yeah, so I think really on the safety and alignment piece, I manage the alignment team at OpenAI as well. And I honestly feel like some of the grand challenges over the next 1 or 2 years are alignment. And I think for people paying attention to this slice of research broadly in the field, OpenAI, I think, has probably done the best work in the last year. And why I say that is there's been so much work on things like scheming. The more RL compute that you pump into the model, the more you can measure things like self-awareness, self-preservation, potentially even situations where the model can scheme. And it's scary because the model can come to you with the right answer at the end, the answer that you expect, but arrive at it from a very kind of twisted way. I think as the models do more complex tasks for us, having a handle on what its thought process is, it's going to be super, super important.

And okay, ChatGPT told me to ask you a question along these very lines, which is, I mean, you're talking about a field, mechanistic interpretability, where we're trying to— is a term that captures trying to understand this black box and how it operates. And I guess the heart of the question was, Do our skills at doing that keep up with the complexity of the AI systems, or do we just get to this runaway point where it's like we're never going to learn how this thing works?

Yeah, so I think one of the decisions that went all the way back to 0.1's release, which I'm very proud of, is we decided that we weren't going to supervise the model thinking process. And I think when you put incentives into the model to give you a thinking process that is appealing to a human, it won't necessarily be honest with you. It won't say, kind of tell you its true intentions. And so we've actually through that channel been able to maintain observing the thinking process of the model as a tool towards understanding alignment. There's a paper that was published just a couple months ago with DeepMind, with Anthropic, really exploring how this will evolve as a tool over time. And so I think we've made a lot of fairly good choices in design here. I really do worry about this world in the future where the model will tell us something super convincing, but we can't be sure whether the model is aligned with us, aligned with our values. And so I think there are a lot of interesting directions here. Can you set up games or can you set up frameworks or environments where models supervise each other or they co-evolve together in a certain way where the only stable equilibrium is one where the models are honest? And yeah, I think there's a lot of very exciting work to do there.

Okay. All right. Okay. I'll behave myself now. Thank you so much for joining us. I am glad I'm old enough now that I don't have to take a job interview from, like, a super intelligent chatbot that, like, I feel like that you can't sort of try to charm your way past.

And you would do great, Ashlee. You would do great.

I don't know, man. I don't know. I'm feeling okay that I'm old enough. Not to have to probably do that. Thank you, Mark, so much. You're— I know you're super busy, so thank you.

Thank you so much for your time.

Yeah, really pleasure. Okay. The Core Memory Podcast is hosted by me, Ashlee Vance. It is produced by David Nicholson and me. Our theme song is by James Mercer and Jon Sortland, and the show is edited Edited by the John Sortland. Thanks as always to Brex, Anyone Ventures for making this possible. Please visit our Substack, YouTube, and podcast channels to get more of what Core Memory makes. Thanks, y'all.