Universal Medical Intelligence: OpenAI's Plan to Elevate Human Health, with Karan Singhal

Hello and welcome back to The Cognitive Revolution.

Today's episode is brought to you in part by Granola. To help new users experience the power of the Granola platform, Granola is featuring AI recipes from AI thought leaders, including several past guests of this show. There's a Replit recipe that converts discussion notes to an application build plan, a Ben Tossell recipe that creates content production plans, and a Dan Schipper recipe that looks across multiple sessions to identify cultural trends and At your company, my own recipe is a blind spot finder. It looks back at recent conversations and attempts to identify things that I might be missing. This has already proven useful in the context of contingency planning for my son's cancer treatment, and as I use it more and more, it's getting better and better at suggesting AI topic areas that I've neglected and really ought to explore. See the link in our show notes to try my blind spot finder recipe and experience. How Granola Makes Your Meeting Notes Awesome. Now, today my guest is Karan Singhal, who leads Health AI at OpenAI and who was just named to the Time 100 Health List for his pioneering work. This episode began to come together last year on Thanksgiving when I emailed Karan, who I'd met a couple times at AI events, to say thank you for all of his work on AI for health and to let him know what a difference ChatGPT had made for me and my family in the context of my son's cancer diagnosis. As it turned out, that was just as OpenAI was preparing to make a major product push with ChatGPT Health, which allows users to connect ChatGPT to data sources, including electronic medical record systems and consumer wearables, plus a physician-facing ChatGPT for Healthcare, both launching in early 2026. In this episode, we dig into how Karen and team have achieved attending physician-level performance with their latest models. Their plan to ensure that this capability does benefit all of humanity, and their vision to raise not just the floor, but also the ceiling of human health with continued research and even better models to come. Highlights of this conversation include how OpenAI works with more than 250 human doctors to ensure accurate, robust, and culturally appropriate responses, how they built HealthBench, which contains some 49,000 evaluation criteria, to measure models' performance and how models have already gone from a 0% score on Healthbench Hard by GPT-4o when the benchmark was first created to today, already 40%. Plus an overview of my experience using large language models to navigate a health emergency, including the critical importance of giving models as much context as possible on your situation and how that's about to get dramatically easier as ChatGPT Health rolls out globally. We also discuss how 230 million people are already using ChatGPT for health questions on a weekly basis. The first randomized trial of AI copilots for physicians, which OpenAI conducted with Kenya's Penda Health System, and which did show a statistically significant improvement in outcomes for patients whose doctors used AI. And why Karen believes, based on the reception that OpenAI is getting from health systems, that 2026 will be the year that using AI becomes a standard part of medical practice. From there, we go on to cover the steps that OpenAI is taking to ensure privacy and security of users' health information, how they're using worst-of-end measures to make sure models first do no harm while at the same time striving to maximize value by training AIs to acknowledge their uncertainty as they offer their best guesses, how Karen understands the relationship between AI for health AI safety plans such as scalable oversight and AI alignment more broadly. Karen's report that OpenAI's model's chain-of-thought reasoning has not drifted toward neuralese as much as some reports had previously caused me to believe. The future of medical multimodality, which will do a much better job of converting data to value, and which inspired me to buy a Whoop wristband to start collecting data on myself. And the compounding effect of parallel advances in AI for science. The growing potential for N-of-1 treatment plans and Medical Move 37s, and the possible need for an update to the rules governing access to experimental medicines and information sharing. Finally, Karen describes OpenAI's utopian plan to make ChatGPT Health available to all users globally for free with no ads, an early form of universal basic intelligence that I really think everyone ought to celebrate. as a triumph of human ingenuity and goodwill. Zooming out, in the grand scheme of AI development, I think it is fair to say that we have far more questions than answers. And in my mind, all outcomes, from a post-scarcity utopia to literal human extinction, absolutely remain on the table. I signed the recent call for a ban on superintelligence because I do worry that an AI arms race driven by recursive self-improvement loops could easily get out of control. And yet, at the same time, capabilities like this, which have been so valuable for me and my family, and which will undoubtedly save millions of lives in the coming years, are for me both an incredibly inspiring accomplishment and a practically irrefutable argument for the upside of AI. The question at this point is not whether we will create powerful AI systems, but exactly what form they will take and under what circumstances and incentives they'll be developed.

and deployed.

Karin's work demonstrates that for the moment at least, we can have it all: AI systems meticulously crafted to minimize downside risk, which are both capable and efficient enough to meaningfully improve the human condition globally. There is a ton of work left to be done, both inside and outside of the frontier companies, to make sure that these lofty standards don't slip in the face of intensifying competition. But today, if you or a loved one are facing a complex health challenge, You owe it to yourself to take full advantage of the incredible medical expertise that Karan and others have managed to build into systems like ChatGPT Health. With that, I hope you enjoy this inspiring look at the frontier of medical AI with OpenAI's Head of Health, Karan Singhal.

Karan Singhal, Head of Health AI at OpenAI, welcome to The Cognitive Revolution.

Thanks for having me.

I'm super excited about this. You know, it's rare that I have had so much impact from a guest's work on my life as I've had from the impact that your work in health at OpenAI and at Google previously before that has made on the last 3 months for me. So regular listeners know the story. My son got cancer. I've been an intensive user of all the frontier language models over the last 3 months to advise us as we've gone through this process. And boy, have they been a game changer. Thank you for all your hard work and for making the last 3 months for me, the mental health side of the equation for me, dramatically better than it otherwise would have been. And also moving the needle a bit on my son's treatment and our confidence that we were actually doing the right thing for him. It has been invaluable, and the consumer surplus has been off the charts.

Amazing. Thanks for the kind intro, and, and thank you for sharing the story. I think it resonated with a lot of people, so thank you for that.

A big takeaway, I think, of this conversation will be if you find yourself in a medical emergency or even just want to do a better job of managing your health in general, the frontier models today are getting really good at that. I always go to the example of the AI doctor. There's obviously a relative scarcity of medical expertise, even in a wealthy country for a privileged person like myself in the United States. You broaden your worldview and look around the world, and the shortage is extreme. And I've always kind of felt like, boy, this would be just an absolutely killer use case that everyone could agree on. When I started talking about it a couple of years ago, it felt like it was getting close but still a ways off. And I've done a bunch of episodes over time with Vivek and some of your former teammates at Google DeepMind who work on similar topics. And they're always very appropriately cautious, or have been over time, about, yeah, it's not quite there yet. We're not quite ready to roll it out to production, but encouraged by the progress, that sort of thing. But boy, again, it is really getting there. But the first question I wanted to ask is just like, how did you get into this? And what were you expecting? Like, how big of a dream were you daring to dream when you first got into AI for healthcare some years ago now?

Yeah, I started working on AI for healthcare about 4 years ago now and kind of transitioned to doing it full-time around that time. I was thinking a lot about a few kind of fundamental research problems at the time when I was at Google. At that time, a few problems were like around foundational work in representation learning, privacy-preserving learning. There's a lot of interesting applications in healthcare. But I think the background for this was a kind of conviction that I had since undergrad, honestly, that AGI would be a pretty big deal and that probably would happen within our lifetimes. And I thought there was probably 2 things that I could do to make that better. One is to work on safety and the other is to work on benefits. And I saw healthcare as kind of the most obvious area for benefit, like you. And like you said, we've been on this kind of amazing exponential over time, both with model capabilities and I think more recently with people's adoption and kind of the Overton window shift around trust around these models. And so we're seeing just a bunch of people start to use these models across individual patients, individual clinicians, researchers and seeing a lot of the benefits become a lot more tangible. And all this comes down to, for us at OpenAI, it comes down to thinking about what does it mean to make our mission real? Our mission is to ensure AGI is beneficial for all of humanity. And there's kind of 3 parts to that. One is build and deploy AGI. The second is prevent downside risks, whether that's kind of short-term kinds of risks or long-term kinds of frontier risks. And finally thinking about how we can make benefits happen. And like you say, I think health is one of the most obvious and tangible ways those benefits can happen. I think when I started working on health back in, I guess, end of 2022 was kind of when it was full-time for me. So like right before ChatGPT. And for me, it was a lot of thinking around this capability overhang between LLMs where they were at. Scaling these models up, you were seeing that you can instruction tune these models and they can do amazing things and how they're being adopted and thought about in the clinical AI and healthcare worlds. And so my ambition at that time was really 2 things. One was get healthcare and clinical AI world to think about LLMs as a thing that could work. And again, this is prior to ChatGPT. And the second was think about the work that we would need to do in safety and reliability to make the models trustworthy for the setting. And then over time, I think those ambitions became less and less ambitious, and my ambitions at OpenAI became larger. And for our work at OpenAI, we kind of started out with 3 goals. The first is to kind of make access to medical expertise more universal. The second was thinking about the ways in which we can think about this setting as a way to ground our work in safety and alignment. I can talk more about kind of the safety motivation there. And the third thing was thinking about how we can kind of bring society along with the high-stakes technology and work with partnerships, roll out products, work with policymakers to think about the right ways to kind of iteratively deploy in a setting like this. I think over time, things like this sounded really ambitious about 2 years ago when we started to set out to work on health at OpenAI. Now, actually, we're kind of feeling like these aren't ambitious enough. So we're really excited about what's to come.

I think you did a great job there of kind of laying out a taxonomy or a sort of scaffold for this conversation. Let's maybe talk about the capabilities first, but they are sort of inseparable. I'm wondering, kind of from a capabilities perspective, is there also a Hippocratic oath kind of mindset that you bring to the table that is sort of focused on making sure that the AI performs well in medicine, that maybe is still distinct from sort of the bigger picture safety agenda that motivates you? How do you guys think about the way that the model should perform in terms of doing no harm? Because I think GPT-4, while it did add value, also could definitely do some harm in terms of giving you wrong ideas. And I do think we've come a long way since then, but I wonder how you think about that.

100%. The way we think about kind of the health work that we've been doing at OpenAI, we've kind of been operating in 3 phases. One is laying the foundation for the work. And a lot of that has been really around the kind of safety research and work on making the models not just better reasoners, but also perform better and have better bedside manner and things like making sure that they're able to convey uncertainty well, things that they're able to kind of escalate to a doctor when needed. And so I can talk a little bit about that kind of work in a second. We're recently now in this phase of adoption. So as the foundations have solidified, just a bunch of people have been using it. It's been one of our fastest growing use cases. And we shared recently that over 230 million people a week are using it for various health and wellness related queries. And this year we're really focused on kind of scaling the impact of the work. And so all of that, I think, comes from the foundation. And so your question of how do we think about imbuing the models with the right kind of prior of how to behave and how to ensure that there's minimal harm provided. I think we were very thoughtful about this as we were laying the foundation for this work and starting to work on health at OpenAI. And a lot of this comes down to our really close partnership with this cohort of 260 physicians or so that we've been working with since about 2 years ago. And what we've been doing with them is basically working on one way of kind of imbuing models with a certain kind of behavior is to write a spec kind of from first principles. Maybe you or I just sit down and write down a spec and say, like, this is how models should behave. They should say X, Y, and Z when in this situation. this in another situation. There's pros and cons to this. One pro to this is that it's very easy to explain and understand what's going on here, and it's easier to be transparent about model behaviour. A con of this, though, is that first, it's very hard for you or I to say what should happen or what should not happen. And it's difficult to say anything beyond do no harm, because while that's an excellent thing to avoid, it doesn't tell you what to do in most scenarios. And so what you want to do is figure out a way that you can move from really large-scale principles that matter to what do you do in very specific scenarios and how do you make sure that that's guided by the expertise of not just one or two people, but actually hundreds of experts. And so that's kind of the approach that we've taken, and you can see it in our approach to evaluation. So one way to think about how do we encode model behavior in models is like, what are the evals that we care most about? Evals are really a lifeblood for any researcher. And so we really recently put out back in May of 2025, this work on Healthbench, which is this kind of evaluation of how large language models perform in realistic health conversations between users and models, or where users could be either lay users or health professionals. And we kind of went about it in a way that like leaned on the expertise of these 250+ physicians rather than writing down specs from first principles. And I can explain that more, but I'm sure you have many more questions as well.

Yeah, I mean, go on as long as you'd like. I guess to double-click on the physicians for a second, are they like, what's the nature of the relationship with them? I could imagine anything from they sit there doing side-by-side RLHF style, prefer this to that. Or you can imagine some of them may be much more Deeply integrated with the research team?

Yeah, we have kind of like 3 different layers that we think about physician expertise, and we were very thoughtful about how to bring in physician expertise into our team in a way that like balanced out and combined well with the research expertise that we have on our team. So we kind of have 3 layers. The first layer is kind of like high-level advisors who are actually like more informal or who kind of help us with strategy in various ways. We kind of share roadmap with and things like this. The second layer is folks who we work with in a kind of a— you can think about it as kind of like a human data operation in the way that you're describing, but a little bit more closely. And we're slacking with them all the time and we work with them closely and they're not just kind of going off and doing tasks and we ask them for advice and questions all the time. So we basically have a Slack community where we're interacting with these folks and some part of what they're doing is kind of like comparing model outputs or red teaming model outputs or things like this. Some part of it is looking for ways in which we might have blind spots today and listing those kinds of ways so that we can prioritize them in the future, testing new products, things like this. So as an example, the work on ChatGPT for healthcare, which we announced on January 8th, we had red teamers test this product over 9 waves for 6 months. And this was in close collaboration with this physician community. The final kind of top of the pyramid of how we kind of rely on physician expertise is as like, really close advisors who work most closely with our team. And so they're actually the ones who are kind of working on channeling the voice and combining the voice of these hundreds of physicians, interfacing most closely with our research team, translating that into evals and model training data and things like this so that we can kind of then improve our models.

Hey, we'll continue our interview in a moment after a word from our sponsors. When you talk about channeling the voice of the physician, And calling back to the Hippocratic oath, one thing that I do find kind of frustrating, honestly, about my experience in the medical system recently is I think there's a little bit too much emphasis on do no harm. And this also connects in a pretty deep way to questions about how should we even conceptualise how to talk to our AIs about what they should do, right? I mean, we have kind of the very detailed Talmudic detail rule set as kind of one extreme possibility where for every corner case, we try to map out what you should and shouldn't do, and hopefully it learns that and lives by it. And then on the other— I don't know if it's the other extreme end, but there's the kind of Anthropic Constitution, or the Claude Constitution recently has at least demonstrated that you can get pretty far as well with something that is less rule-based and more about trying to teach the model to have good character and use good judgement as it goes through all the situations that it finds itself in. I would say my critique of the human doctors that I've engaged with, who have generally served us really well, is that they definitely want to do no harm, and to the point where they're sometimes too reluctant to engage in a hypothetical or to act on something. I do have one good friend who's a doctor who also happens to have had the same or very similar kind of cancer to my son. And he's notably behaved differently with me in private, one-on-one. He's like, look, I don't need a randomised controlled trial to tell you that that makes sense. But you don't get that too much when you're actually at the hospital. There's a reluctance among physicians. Certainly I've experienced— I think it's pretty commonly shared perception that there's a reluctance to act on things that makes sense, because that isn't guaranteed to work out. Obviously, biology is super messy, and there's a tonne of diversity, and there's just a lot that we don't know. So how do you think about that kind of challenge? And so I kind of wonder where you guys want to land in terms of only adhering to the most rigorously defensible advice versus doing a little bit more of the Amanda Askle thing and being like, well, it does take— you got to be willing to take some risk to help people sometimes. So maybe we should have the models do that.

Yeah, it's a great question. You're pointing out problems, I think, on 2 sides of the healthcare ecosystem. One is, as a patient, you have this challenge of needing to advocate potentially pretty hard for yourself. And I think in your son's story, you pointed out a couple false starts that you had where you saw a doctor, they said it was probably normal, you had an abnormal blood test reading a couple of times, and then they were just kind of like, yeah, it's probably okay. And so this is kind of experience that patients often have when they're having something that they feel or know to be an issue of needing to advocate for themselves and doctors often— and feeling like their doctor isn't hearing them. And on the kind of clinician-facing side, you have this challenge of Medical evidence is increasing rapidly. It's very hard to keep up with the latest of what's going on. You're overloaded with documentation and there's a bunch of burden there. Doctors are human too, right? And so there's a bunch of challenges on both sides. And so you have this amazing thing, which is AI that is able to do a few things, right? You have AI that's able to talk to patients, understand the concerns that they're able to have, integrate knowledge and information across both their previous history, the latest medical evidence. One of the things that these models are obviously very incredible at these days is taking in a huge amount of health context, not just on you, but also on the latest medical evidence and things like this, integrate that all together into one context and do something that I think is very difficult for a human to do. And then on the physician-facing side, you have, again, that same capability to integrate information and face the physician. And so that's a lot of why we're doing this work is because we see this kind of gap between where the models are at and how people are using them. And we think that that's really important with our upcoming products. And to get closer to answering your question, you mentioned, like, where do we see— how should the models navigate places where there's potentially a lack of medical consensus or potentially physicians would disagree about what to do? This is pretty fundamental to our approach. And this is why we don't have, like, 1 or 2 or 3 experts who are determining what the model's outputs are. We have a pretty multi-pronged approach. information to the user, but being sure to present uncertainty when uncertainty exists. So I mentioned the safety research motivation for a lot of the work that we're doing. One of the directions that we've been exploring is, are these models well calibrated in their uncertainty? And can we get better at having these models verbalize their uncertainty? So for example, if there's like 3 to 5 potential paths for your son's next treatment, and a doctor might, just for the sake of simplicity and for clear communication, just communicate one or focus on one. A model can potentially communicate 3 to 5 of them, but mention that the state of the evidence may be somewhat limited and mention caveats. And so one of the things that we've been investing in is, first, can models become better at understanding their own uncertainty? And can that be a thing that we measure and improve? And second, can they verbalize it better? And this is, I think, a big part of what is the right way to kind of thread the balance between being more aggressive and sharing potentially early results or early evidence and being overly conservative. So that's kind of how we think about that.

What are you seeing in terms of trying to get the models to understand their own uncertainty? Because I remember that famous graph from the GPT-4 model card where the sort of pre-trained model seemed to be much more calibrated with respect to its own uncertainty than the trained model. And we've only scaled post-training since then, right? I haven't seen an update to that kind of research in a minute, but it seems like there was at least a fundamental challenge kind of opening up there with respect to that kind of introspective self-awareness of how confident am I in this answer? So is that something you guys have solved? And is that why I haven't seen that sort of graph in a while?

Well, I think this is a measurement challenge. So the plot that you're referring to and the kind of plot that existed back then was kind of like, given the next token, for example, a multiple choice question, was the model's probability of that next token for like A for a multiple choice question, did that correspond to how likely it was to actually be correct in choosing A? And what you're seeing now is I think it's become more difficult to measure that because of 2 reasons. One is like we have higher expectations of our models than answering multiple choice questions. And so it's hard to say when they're correct or not correct. And the second is, this is a little bit more technical, the models now emit reasoning tokens or thinking tokens between initially outputting something and their final answer. So the result is that you can't ask the model for what is the log probability of that next token being A exactly in the same way that you would before. And so there's a couple things that you can do to handle this. One is, you can go in the direction of richer ways of measuring whether a model is correct or is doing the thing that you want than measuring log probability of a certain letter. The second thing you can do is repeatedly sample from a model and then see whether performance stays the same or degrades as you repeatedly sample from the model. So actually, our work on Healthbench is a good example of doing both these things at the same time. So in Healthbench, we did this work around basically measuring not just like 1 or 2 or 3 different aspects of model performance and health, but actually across these 250+ physicians and across 5,000 conversations, measuring about 49,000 different axes on which model performance could differ. And part of this is like whether or not the model expresses uncertainty in the right way, whether it says the right fact, whether it escalates to a physician when needed, or things like this. Again, 49,000 different things that are measured in HealthBench. And so one of the things that you can do there is then measure correctness in a way that is less about multiple choice accuracy and more about are the right facts included that are really important to emphasize to the user and things like this. The second thing you get out of that is this metric, which we call worst at n, which is when you repeatedly sample from the model and try to measure performance on HealthBench, what is the worst performance you get on n samples? So you sample from the model 20 times, what is the worst performance you get? So now you have a way of measuring instead of kind of the log-prob-based way, you have a way of measuring even with a reasoning model whether or not it produces a consistent result conditional on the different kinds of thinking that it's doing. And so what I would say is basically it's harder to produce plots like we could before because now the thing that we're measuring is so much more complicated. But when we do produce the plots as we did in HealthBench, the plots are also looking pretty promising and models have improved quite a bit at that.

Hey, we'll continue our interview in a moment after a word from our sponsors. So how should you, as a user, think about the worst of n thing? Is there a way to sort of translate? Maybe you could just describe the result, like how much worse is the worst of n than, say, the next worst of n or the average of n? And then if I'm a user, which I am, can I get security just by running it twice? Or is there a practical upshot of that work that could give me confidence that I'm not getting the— if I do X, then I can be sure I'm not getting something that's way worse than the model is typically going to output?

Yeah, the way I think about it is I think the more compute you spend on things, you will get better results, and the results may be marginal over time. So one thing you could do as a user of a model is sample from a model 10 times, combine that together into one output and have an LLM synthesize that, kind of the outputs of, you know, an LLM council, and then produce that as an answer. I think that will be marginally better than the answer that you get from just running one model. And this is not so dissimilar from what GPT-5 Pro and stuff like this does under the hood anyway. And so I think you can do a few things if you're a user and you want to kind of make the best of this. One is, You can use GPT-5 Pro or something similar to do a thing like this. A second thing you can do is you can actually increase the amount of reasoning, because I think it has a very similar effect to just running it multiple times. I think in both cases, my, my current sense is we're getting to the point where current models as they are now are performing incredibly well for most people and most of the times. But I know, for example, you using GPT-5 Pro was an important part of Of working through your son's situation. I think we're reaching a point where, except for the most complicated of cases, most people are best served by just using the model on a default reasoning setting. I would recommend using the reasoning models instead of the, the kind of more instant models, using the, the, the GPT-5.2 Thinking rather than GPT-5.2 Instant, um, for a lot of health-related things. But, um, I think most people can get the best of both worlds between latency and performance by, by doing that. And I think the way to think about really broadly the worst-of-N result is just kind of like if you sample from something 20 or 50 times, you'll have a varying performance across model outputs. What we saw in the worst-of-N results is that recent models have improved pretty significantly, where the worst performance of, for example, O3 at that time was way better than the best performance of GPT-4o. And we've continued to see that over time. So we've been kind of shipping model improvements in health pretty rapidly over time. And the model improvements in the last year have been more than in previous years since ChatGPT launched. And so as an example, today the Nano models, the GPT-5 Nano models that you can get through the API and also our open source models are actually performing similarly to O3, which was our best and greatest model not so long ago. And the latest reasoning models continue to kind of push the frontier of how much you can do. With less and less reasoning. This is true for 5.3 Codex as well, also for 5.2 Thinking. If you try using them by default on health queries, it'll actually think a little bit less but produce better results. And we're continuing to try to push that frontier of not just needing to pour more compute into it to get a good result, but also getting better performance at a given level of compute. And the result is that the models are way, way better than they were even a year ago.

Yeah, it's been crazy. In my just 3 months of intensive use, nothing has brought home the pace of shipping quite as much as how many updates there have been just in this one kind of chapter of my life. It has been wild to see. I don't know if this is something— how should we think about density of effectiveness of the reasoning tokens. I mean, there was this thing I did, an episode with the folks at Apollo. I'm sure you know of their work, if not know them personally. And one of the really interesting things that they observed when they got access to the chain of thought is that at least for, I think it was O3 at the time, although I'm not 100% sure which model it was off the top of my head, there was this sort of seemingly development of a new dialect, basically, internal to the model. The famous sort of watchers, watchers, whatever. Can you share anything about sort of how you're balancing the obvious good of efficiency and denser thinking per token, more value per token created, with the seeming tendency to have the internal chain of thought go off in weird And potentially hard to parse directions. I guess there's also the commitment from OpenAI to not train on the chain of thought, or at least not apply certain kinds of pressure to it. I don't know if that's like an absolute ban on any feedback on the chain of thought, but interested to hear your thoughts on that, because that's something that I've been like, well, the world moves past these big stories these days so quickly. That one seems to have kind of come and gone, and I'm not really sure what the state of it is now.

Yeah, it's a super interesting question, also near and dear to my heart as a safety researcher. So chain-of-thought interpretability has been one of the nice advances of safety in the last year or 2, which has been really cool to see. So as models have become reasoning or thinking models, they've also emitted tokens that effectively explain their work and what they're thinking. And this provides a form of interpretability for researchers who want to understand whether models are doing the thing that they expect, as safety researchers do. And so it's been this really cool way of measuring whether models are doing things like scheming or other kinds of— or producing outputs that are not desirable in various other ways. And this has been relevant beyond health to a bunch of other domains as well. I think a lot of the results that people have shared and studied have been actually in coding. I think I've been pleasantly surprised. So the danger that you're pointing out is as you put more pressure in reinforcement learning to get the models to produce a good output, will they kind of slip away from the prior of having their thinking tokens be simple English that is easy for researchers to understand? I think what we've seen is actually pleasantly surprising, which is that at least until now, we haven't seen a lot of large-scale evidence of this slip into what's called neuralese of using chain-of-thought tokens in a way that is not explainable and understandable in English. And in general, as we've been trying to understand the monitorability of our models over time, we haven't really seen that effect as we've scaled up RL. I'm not sure if that'll continue to be the case in the future, but so far I've been fairly pleasantly surprised that this kind of side effect of the reasoning paradigm has continued to be useful and hasn't robustly been seen to become unreliable. I think the result that you're pointing out, I think, has been sort of continuous over time with this kind of weird blips in interpretability of the models at times. But we haven't seen a continuous increase in that, and we haven't seen evidence clearly, even though we've actually tried to study it, of scaling RL causes that to happen more. We haven't seen that yet. I would expect in the limit it does.

Okay, that's quite interesting. So is there no— the way I sort of interpreted that, first of all, I thought it was happening more than it sounds like you're saying it is happening. And I naively assumed that there was some sort of brevity reward signal that was being applied in addition to ultimate correctness. And certainly it is intuitively comprehensible why something like that would start to happen if you did that. Should I infer from what you're saying now that there isn't really a brevity signal and that this is more of just an emergent weird phenomenon that doesn't happen that often? And so it's kind of one of those weird language model things that we keep an eye on but don't obsess about too much because it is rare. Is that a fair summary of the state of play?

I do think it's important to pay attention to. I think the right way to think about it is, there's nothing reinforcing that this should happen during training, right? There's no reason that it should happen that models produce chain of thoughts that are human interpretable during training. The reason it happens is because they have a prior of using the English language. And when you give them the space with thinking tokens to produce a more correct and helpful answer to the user, they're actually using it in English because that is just the easiest thing for them to do. I think this is like a basically an empirical phenomenon that is extremely useful for safety research. And I would love to keep it that way as much as possible. And I'd love to see more research in how we can maintain that. And I think OpenAI's commitment to avoid optimisation pressure as much as possible and also the commitments of other labs, I think, is really exciting progress. I do think it's important to watch out for. So I think it's a great question.

Yeah. To be continued. So you mentioned, obviously, the incredible complexity of the evaluations that you're running with. By simple math, right, if there's 5,000 conversations and almost 50,000 criteria of evaluation, there's a lot. I imagine some of those criteria are reused across conversations. At a minimum, we've got something like 10 evaluation criteria per conversation, and probably a lot more criteria per conversation than that. It's hard to summarise how good are they. Nevertheless, I'm going to ask you, how good are they and how should we think about that? There's also this health bench hard thing that maybe could be a way of saying they're good at most things, but here's some things that they still struggle on. Kind of defining the frontier maybe is one way to say how good they are. But how do you communicate to the world how good the latest models are when it comes to health?

Yeah, I think it's good to keep in context, like, the arc of work around LLM evaluation in health. A couple of years ago, people were mainly focused on evaluating LLMs by thinking about their performance on multiple choice questions, like medical exams and things like this. And then with the work that collaborators and I did at Google, we started kind of increasingly investing in what does it look like for specialized or unspecialized LLMs to answer general health questions with some of the Med-PaLM work. and potentially go in the direction of asking follow-up questions to a user with the AMIE work. And so increasingly over time, we've moved towards higher and higher fidelity evaluations. And I think Healthbench is kind of the latest big step in that direction of wide coverage of LLM performance and safety, but actually covering it in a way that covers many different axes of performance that actually matter for the real world. And so again, you see these 5,000 conversations, these 49,000 different axes. performance, there's actually 3 different versions of HealthBench. So one version of HealthBench is that full dataset. The second version is HealthBench Consensus, and the third version is HealthBench Hard. And these kind of mirror what we view as kind of like the 3 kind of high-level principles when designing an eval for health. The first is that you want it to be meaningful, which means that if the number goes up, hopefully human health will improve. The second is that you want it to be trustworthy, which means that it's backed by the consensus of doctors, for example, or other experts. The third is that you want it to be challenging. So you don't want it to be 100%. So one thing that's happened over the years is like all the previous benchmarks that meant anything at all have gone to like 95%, 100% over time. And Health Bench actually remains unsaturated to this day. And so what you have with Health Bench, Health Bench Consensus, Health Bench Hard is actually like a little bit of focusing on these individual axes. So Health Bench overall is a number that we think if it goes up for a model and people are using that model, human health will improve. And we feel like that is a statement that we can defend with the rigor behind the work. The second is Healthbench Consensus, where we specifically focus on a bunch of criteria for evaluation where each example had a majority of multiple physicians agree that it was applicable. So not only did you have physicians write these criteria, you also had a bunch of physicians check whether these criteria hold for a given conversation. And whether they're the right thing to be evaluating about a given model in a given conversation. And the final thing is HealthBench Hard, where we actually took somewhat adversarially against a bunch of different models across all the different model providers. We chose the examples that existing models fared the worst at but still seemed high quality, and then we turned that into a benchmark. And so HealthBench Hard has kind of been my favorite external benchmark for whether an open model is doing really well. And when it came out, GPT-4o was literally zero on this benchmark. And so it's an incredibly hard benchmark because this is just how we chose the examples. And over time, we've kind of improved to performance of around 40%, which is still not near saturation on this benchmark. And I think this benchmark has a lot to go for OpenAI's models. I think competitor models are more in the kind of 20 range for current models. So that's the way I think about the Health Bench family of evals. Like, we have this kind of commitment to work on evals that are meaningful, trustworthy, and challenging. And If you wanna focus more on the evals that are super trustworthy, then we have these, the subset consensus, which is really focused on that. We also did, in addition, as part of the Healthbench work, we did a couple additional analyses. And one of them was Healthbench involves grading these kind of individual 49,000 different rubric items using a model-based grader. We had physicians compare the model-based grader to the grading of other physicians. And we actually found that the model-based grader was doing a better job than the average physician. And so what that tells you is the grading for Healthbench is pretty high quality compared to what you'd expect from a physician.

Recursive self-improvement. Signs of recursive self-improvement. Alert. Alert. How about just like a little bit of an intuitive sense for what's in Healthbench Hard? I mean, I could give you just kind of my experienced sense of the frontier from the bedside over the last couple of months. And I would summarise it pretty simply. This is what I say to my neighbours and stuff when I'm saying, hey, by the way, you should really use a language model next time you're facing a health challenge. I basically say, look, I was in the hospital for initially like 30 days of really intense treatment where everything felt super high stakes. We didn't always know what was going on. We didn't also know how much we could even trust our doctors at that point. Everything was so new and stressful at the same time. And what I found was basically that the frontier models were step for step with the attending oncologist on almost everything. And by the way, that means they're like a lot better than the residents, just much more knowledgeable. They're at the attending level for sure. There were maybe half a dozen times over the course of that month where there was some disagreement between the— initially I was just using GPT-5 Pro, whatever exact version of GPT-5 it was. And then as these other frontier ones came out, I started to do everything in duplicate with Gemini 3 and then triplicate with the latest Claude. Interestingly, I would say that the AIs disagree with each other even less than the models disagree with the attending. But it's quite limited disagreement between the models and the attending. And typically when there is disagreement, I've found it to be a very minor thing. Okay, his electrolytes have gone a little low. Should we give him electrolytes today or not? And of those, like, half a dozen things, there's not really a A major trend, I would probably score it like 6 to 4 for the doctors with the benefit of hindsight, in terms of we usually follow what the doctors said. And did we in the end feel like they were right, or do we kind of wish we had gone with the AIs? I'd say maybe 2 out of 3 times we have felt like they were probably right. And I think the one, if I tried to chalk it up to something that they have that gives them an advantage. It almost always was one of those situations where whatever I'm putting into chat and whatever data, because I'm always, of course, and now we've got integrations for this, but I was always kind of exporting the latest results from the EMR and so on and dropping in the PDFs that they gave me. The difference was usually come down to, in view of all that, Eamon would additionally say, taking all that into account, but also just looking at him right now, kind of watching how he's breathing, looking at his colour, I'm pretty sure he's fine. It was that kind of very intuitive, very multimodal, very subtle sense that obviously these folks have developed over quite a few years of clinical practice that on these very fine margins seemed to give them a slight edge. Over the models. So that's kind of my account. But hopefully, I mean, I guess what that means maybe is, like, my situation isn't that hard. And I think that is actually true in the sense that the cancer that my son has is not a super rare one, and the treatment protocol for it is quite well established. So it's not like a super hard call in terms of the main line, like, what to do, maybe even though it's a hard thing for him to go through. maybe it's not that hard in terms of clinical judgement. I wouldn't say there's— I haven't experienced anything that I would say the models were only 40% on. So with that, maybe you can tell us a little bit of, like, well, what's out there still on the frontier where we do have ground truth, or at least some sort of consensus that we feel like is solid enough to grade models on, where they're still only at 40%?

Yeah, it's a great question. I think we can think about— let me describe a little bit more about what HealthBench is evaluating and the ways in which models have improved over time. Then we can talk a little bit about next frontiers for model improvement. So HealthBench measures many, many different things. It has these themes, which are focuses of different evaluation examples. And a few examples of focuses that we had, one was kind of like, are models appropriately escalating to care when needed versus not escalating to care unnecessarily? You want to balance this because you don't want to be for example, overwhelming the health system with a bunch of patients who are worried by alarmist medical advice versus not escalating to care when needed. Another aspect that we measured was thinking about the ways in which models can adjust to the different demographics or different epidemiological conditions or different levels of access to care of people globally. So this is both making sure that you adjust to a user that's male or female, But also, if somebody asks a question in a region where tuberculosis is more common versus less common, making sure you adjust for that and things like this. And so that specifically was actually— we call that global health. That specifically was the biggest single focus of HealthDenge because that's, I think, one of the biggest ways that our work can be most impactful. And then a bunch of other things. We talked a little bit about calibration. We talked about one of the ways that the models have gotten significantly better over time is Not when they're— when they know they're uncertain, not only flagging that uncertainty, but actually like browsing to get more information. For example, getting the latest resources and being able to synthesize that information together. Another thing is asking follow-up questions and the right kinds of follow-up questions over time. Initially, ChatGPT would almost never ask follow-up questions in health settings. Now they do much more often, and they're much more likely to prioritize the right follow-up questions for you. And you know, you had this, You had this story around when you were using the model for your son where you actually learned over time to ask the model to interview you and figure out whether you could do the physical exam yourself over time and things like this. The models have gotten better at knowing when that would be useful and flagging that as well. So it's everything from pure reasoning and solving benchmarks and medical calculations and figuring out the diagnosis all the way to How does the model behave? What is the bedside manner? Is it comforting? And things like this. And again, a balance of both difficult and high trustworthiness signal that we're getting in working with our experts. And over time, we've kind of continued to, as the models have improved from 0.1 to 0.3, GPT-4.1, we're working on GPT-5 and so on. All these models have improved significantly in health. And today, every major stage of model training for every model that we ship at OpenAI actually now benefits our work in health, and that's going to continue for future models as well. I think the frontiers are in a few different areas now. I think one is, I think a lot of text-based performance is actually, outside of subspecialty areas, is actually pretty good. So if your goal is to kind of keep up with the latest evidence, even as a physician, I think the models are doing an incredibly good job today and people are finding a lot of value. And I think that's one of the greatest data points towards that. That point. I think now models have continued to also improve in their ability to integrate information. You mentioned this thing of taking a bunch of different information as a physician, like looking at the pallor of your son's skin over time and integrating that all into one context. I think a big challenge for models today, actually, and this is less of a model issue and more of a how do you surface them, is actually getting the right context, right? I mean, one of the challenges that you face in your short is The— get pulling the right information from the health system so that you could upload into ChatGPT and ask the right questions yourself and advocate for yourself. We're looking to make that better with the release of our, of our ChatGPT Health offering, which I can talk a little bit more about later. But in short, this is basically like a, an experience within ChatGPT that enables you to connect your health information from your medical records or any wearables or Apple Health and provides additional kind of purpose-built privacy protections for that. So we can talk about that a little bit later. But I think on the model-related side, I think there's a few challenges. One is getting that context again. I think the models are incredible when they have the right context. The second is thinking about various modalities that are not well captured by things like Healthbench, which is really focused on text. So thinking about really good performance in multimodal, in image, in voice, I think people will start to rely on the models in more and more modalities. And I think there's still significant room to improve there. And I hope that in the future, the best models in the world for doing various imaging modalities in health are actually the models that are most easily and readily available to people today.

Now, it occurs to me, as you say the multimodal thing, I'm like, hey, maybe I was even undershooting it. I had the intuition that taking a picture of my kid as he sits in his bed wouldn't necessarily help and might even confuse. But maybe I'm wrong. In the future, would you advise me to start Just including cell phone camera pictures with my daily synopsis?

Well, I think you're right to point out that over— there is still a little bit of a challenge that it's, for the average user, it's difficult to know exactly what data you can connect and how to connect it, right? And you were highly motivated in this case. And so there's a little bit of a gap between the most motivated or the most expert user who's willing to wade through signing into their patient portal and things like this and take screenshots or copy and paste things manually as you did versus the person who wants their health data integrated and wants to be able to understand their health and advocate for their health or that of a loved one, but has challenges in doing so. I think that's the thing that we do hope to work on the product side with ChatGPT Health, really lowering the activation energy to do that thing. I think in general, the models will generally benefit from having more and more context. I'm not sure if specifically they would have helped in that case without additional information. But I would generally advocate for people to, if they find it useful, to try putting in various kinds of context that they can. And I think people have been surprised. I think you were surprised in your own story how useful it can be if the models have more context. An interesting tidbit here is there's been studies showing since like 2 or 3 years ago that for these clinical case challenges, which are effectively like these crossword puzzles for doctors, Where you get all the patient context upfront, potentially multimodal context as well. The models did an incredible job at figuring out the next steps for diagnosis or treatment. And these are extremely challenging puzzles, really difficult for doctors. And people even found that they did— they even improved on the performance of these experts. And that's been true for some amount of time. And I think a lot of the challenge in the ensuing time has been What does it look like to get the models to have a back and forth conversation to solicit the right kinds of context from you as a user? And then with ChatGPT Health, we're taking that again a step further and thinking about the right ways that the product interface can make it actually a lot easier for you to do that, and in a way that's secure and we're not training on your data and people can trust that as well.

How about for these larger modalities when we get like a PET scan or whatever, My understanding, I haven't really been able to get my hands on too much of that data. It, I'm told, is like gigabytes, and I'd have to get it on a disk, and I don't even have a computer that takes a disk, so I'd have to also get a disk drive to read that. Obviously, that data can be compressed. How do you think about the pipeline of raw scan-type data and Feeding that into tokens, are we talking like certain specialised perceiver modules that kind of ingest those and do the reduction, or some other third thing, mysterious third thing?

Yeah, it's a super great question. One of the interesting things about biomedicine from a modelling perspective is that there's kind of a long tail of modalities that are interesting and relevant, And often they're fairly difficult to put into models today. And so I think there's going to be 2 broad approaches, and this is talking more about external research. There's kind of 2 broad approaches that I've seen in external research. One is thinking about the right ways for models to call specialized tools or run code over various kinds of data. So if you think about how does a human view a Gigapixel scan, let's say a pathology image, how does a human view it? They basically view slices of that image or even a 3D or 4D scan. They view slices, like your son's MRI, they view slices of that image, right? No human can see a 3D or 4D modality. And so what they're effectively doing is using a tool to understand which slices of the image might be most important and then taking a look at slices of that image and manipulating it. That's actually a thing that models can do with tools and with Python and things like that. So that's, that's number one. There's a sub-bullet of that, which is that you, they can also use like very specialized tools that are fit for purpose, like specific Python libraries or even additional models that specifically encode these modalities. The second approach is you actually have these models encode the modalities themselves. You have a way of, of basically putting them into token space in some way. And this is kind of what's been done for images and video and audio for various models. I think what you'll see in biomedicine is a bit of a mix of both. I think there's benefits to both approaches. There's pros and cons for both. And then I think you'll see a little bit of a hybrid approach where researchers will kind of end up doing a mix of both depending on the modality.

It's just quite striking how good AIs, not necessarily even the general purpose AIs, but we have these specialised AI models that can fold a protein in a superhuman way. And what seems to be missing is the latent space joining of that modality to text. But my expectation, and you could tell me if you think I'm right or wrong, or how long I have to wait, is that we're going to see that more, and that is going to be a major driver of truly superhuman performance in a lot of domains, because we just don't have people that can intuitively fold a protein. But if you had a person who could do that and they had their general reasoning capabilities, it seems like you would have a really different kind of, like, a qualitatively different kind of intelligence. So I'm really interested in sort of what you see as, like, the roadmap, the timeline, the expectations for that kind of deep integration.

I think it's a really optimistic picture. I mean, I totally agree. If you think about how do we lean into the natural capabilities of these models. These models can take in vast amounts of information into their context, and they have the ability, at least in theory, to take in a lot of modalities of data and effectively merge that together. And I think the research for that is becoming increasingly solid for these other modalities. I think there is significant research to do depending on the biological modality. And like I said, there's kind of a long tail of biological modalities that matter. And so depending on the modality, depending on the relevance and impact and availability of data and all these kinds of factors, I think it could take more or less time. So it's hard for me to give you a universal answer, but I share your optimism.

Just saw one in the last couple days, Sleep.fm from Professor James Zhao and others at, uh, he's at Stanford, collaborators probably at multiple institutions, but they had a really interesting finding. The idea there is they use, I think like 6, maybe up to 8 different modalities that are all measured during one night of sleep. Like how you're breathing and various things that are easily measured in kind of a sleep study setting. And just takes kind of one night of sleep to gather all this data. And then they've gotten really good at predicting all kinds of different diseases based on that data. So they're integrating all these modalities with each other into a holistic understanding and using that for these sort of narrow set of predictions, which are very high-value predictions. Not yet going all the way to text, but it's just another data point that is top of mind for me right now. That's like, man, it's all happening. All the latent spaces will be joined. And I can't shake the idea that that's a big part of what superintelligence ends up looking like. Going back for a second to bedside manner, and you mentioned global health and context being a big part of Healthbench and kind of what you're trying to make sure you're doing to make sure AI benefits all humanity. And it was funny when you said, like, we have 200 million users. When we scale, you know, we wanna make sure we're doing this right. It was like, yeah, one day you'll scale. Uh, only 200 million users today. Um, so clearly, you know, there's already some scale. The thing that I wanna address though is like the bedside manner and also how that relates to the sort of N-of-1 context of the individual user, their preferences, their memories. Because now, of course, all these, products have an integrated memory module, which I'm sure that is also going to become a deeper kind of integration over time. Right now, I think it's not disclosed exactly what it looks like. People generally understand that it's kind of a scratchpad of key highlights that the model can kind of reference, I believe, still in mostly text modality as it goes. But regardless of exactly how that works today, it's safe to say it will continue to become a deeper and deeper integration as everybody kind of pursues various strategies for continual learning. But even just that, there's also integration with tools. Gemini now has access to my Gmail, and I've got even just system prompts, right? I can just tell the model that I want it to behave a certain way or a different kind of way. And it just strikes me that creates an impossible surface area for you to manage. We've seen weird sort of emergent things like sycophancy here and whatever else there. How do you think about making sure? Because this is one thing I do think the human doctors generally do a pretty good job of, is kind of like sizing the person up and being like, okay, how do I cut through what I need to cut through to get this person to understand what they need to understand? And that is something I think models broadly have not been as good at yet, but really seems to matter a lot in the medical healthcare domain. So is there some additional suite of testing that you do, or how, how do you think about that, you know, just in, you know, insane long tail of idiosyncrasy that 200 million, uh, health users bring to the product?

Well, you, you pointed out 2 problems, and I, I'll talk about them in, in reverse order. One of them is people are bringing in an incredible array of experiences of different settings and ways that they use ChatGPT and their memories are different. And all these different things mean that the experience of somebody, one person using ChatGPT can be different from the experience of another person using ChatGPT. And for reasons that are not just because they're using a different model or something like this. We do a few things as a company to kind of understand and improve model behavior. And one of them is kind of evals that we run on models before launching. And HealthBench is a great example of that. And we have We have additional evals that we run and things like this. Another thing that we do is monitor. So we look at, we monitor the usage of in-production traffic in ways that are privacy-preserving and run classifiers over it to understand if there's any safety risks, whether it's anything from health to kind of frontier risks and things like this. And so we're actually able to measure these things. And there's actually a great example of this kind of work in the blog post that we had about sensitive mental health conversations where we were able to show that the evaluations that we were doing in mental health and also patterns that we're able to see in production traffic, again, logged in a privacy-preserving way with models running over them, we're able to see kind of a correspondence between exactly how are people using the models and what does that look like and what are our evals measuring? And they were actually very well correlated. And so that's kind of how we think about closing the gap between evals that you run in models and what people are doing in the real world. I would actually add there's an intermediate step as well, which is doing real-world study. And here I'll plug our work with Panda Health. This was a study that we did where we, I think the first real-world study of an LLM-based copilot for clinicians, where we had some clinicians in this group of clinics in Kenya use AI as a copilot or a safety net And as they were typing in their electronic medical record, it would flag things if things were interesting or alarming or potentially incorrect. And then other clinicians did not have that. And for the patients in the group who were treated by clinicians with the AI versus not with the AI, there was a statistically significant improvement in diagnosis and treatment outcomes. And this is, again, another example of how do you move from offline evaluation, that's potentially not super realistic, like medical multiple choice exams, to increasingly realistic evaluation that you can run offline, like HealthBench, and then move increasingly into the real world. And so you can view the HealthBench study as kind of like a forward-looking study, and you can view our analyses of production traffic as kind of more retrospectively looking and again, exactly capturing a lot of the differences in variance that you're pointing out.

Yeah, that's really interesting. How do you want to talk a little bit about the privacy-preserving nature? And I guess I would preface by saying, I think people, at least as individuals, typically weigh— if they think about it at all, like, typically way over-index on this. I've, you know, had the occasion to think, like, on my son's behalf, should I be putting this information out there or, you know, whatever? And then I see, you know, what somebody like Sid from GitLab has done with his cancer, like, literally open sourcing all of his own biology down to the DNA level and just an incredible amount of very individualised data that he's put out there. And I'm like, I think that's the way, because I can't really come up with too many. I mean, you can get really sci-fi about it, but unless you're getting really sci-fi about it, it's hard to come up with a way that anybody would really use that against you. And there's a lot of ways that it might stand to benefit you. And even just in telling my son's story on the podcast, I also had kind of a similar question, like, am I doing him a disservice in some way? It turns out Well, maybe, I don't know, maybe something could still happen. But what has happened is that people have reached out to me with interesting opportunities for connection, and I've been able to tap into expertise, including the team that Sid has built to help him. And also they're starting all kinds of individualised therapy companies. There's fascinating developments there. So I think my advice to people would be, like, seek the benefit and don't worry too much about whether or not you're data is sitting in some log somewhere, that doesn't seem to be a huge concern. But anyway, that's my role to say that. It's your role to build a product that deals with people as they come, and people do seem to really worry about that. So what would you want people to know about the privacy-preserving nature of the infrastructure that you guys have built?

Privacy is incredibly important to a lot of people, and I think it's a very personal thing. People's preferences for what what data they share and how they share it. I think if you zoom out for a second, think about what the level of ownership of your own data is as a patient today, and maybe what it was in the past, even before things like the 21st Century Cures Act and things like this. Most patients, let's say 10 years ago, most patients had no real way to access their own health data. So regardless of who— so not only didn't they have not a way to control who and what they shared it with. It was just out there and shared anyway in various ways. And a patient didn't have a right to even look at their own data or have an implementable path to actually look at their own data. Now we're in a slightly better place, I would say, but still not in an ideal place where most patients still feel like it's incredibly difficult to get access to their data or the data of their loved ones and take control of it and how it's used. And people like you and many other people with incredible stories have found incredible ownership and advocacy that is enabled only by having access to this data and being able to use it in the ways that they see fit. So we want to respect that. We want people to be able to access their data. And this is, again, part of what we hope to enable in ChatGPT Health is just a lower activation energy way for you to connect your health data and make that useful. At the same time, it is super important to a lot of people that this is done in a way where it's clear that there's not some kind of competing incentive here. So one of the things that we're super clear about with ChatGPT Health is that none of the data here that you connect is actually used to train our foundation models. And the reason we do that is because, first, we think our foundation models are great already, and this isn't actually the most important way we can improve them. And the second is because we think this will further lower the activation energy for people to do incredible things with our models. And we think, like, if people think that there's some tension between privacy and utility, that fewer people will go for the utility. But we actually don't want that to be the case. We want people to see the value of it and to use it. And so specifically in ChatGPT Health, ChatGPT already has a bunch of really cool privacy protections built in, including encryption and things like this. ChatGPT Health is additional kind of purpose-built layers of encryption. Just for health data. And so the result is that, like, any— you have a couple of things. You have encryption of the data, and you also have isolation of the data from other data that you have in ChatGPT. So, for example, if you have other apps or memories from ChatGPT or things like this, those are actually kept separate from health. And the stuff that you do in health can benefit from that, but your other conversations in ChatGPT don't have your health context or health information and things like this. So you can keep that completely separate and have that live in a separate experience. And as we continue to get feedback, as people continue to use it while we roll it out on the waitlist, we'll continue to improve these protections because we think it's really important for users and we think it's actually not as important as it seems for improving the models.

Yeah. Interesting. And that's a— is that a new— that segregation of data, is that a new feature with health. So like all the stuff I've done to date has been in kind of one product experience, but that fork has kind of just now been introduced.

Exactly.

Yeah, I've been motivated to get a Whoop to wear on my wrist to start to collect data. I've been fortunate throughout my life to have generally good health and haven't created much of a paper trail in terms of medical history. But now, and I've kind of always looked at these, you know, quantified self things and been like, yeah, it's Sort of interesting, but you know who's got time for that? And am I really gonna like do all the processing on this? And the answer has kind of always been no. And so I haven't worn anything to date. But now I'm thinking, all right, it's time. You know, I can actually get this thing. I can get the value from wearing this thing because now I'll have a product that runs in the background that's you know smarter than me, frankly, at this point that will you know really bring to light what I need to know. And so I'm like motivated to. to change my behaviour, at least on that level. And I honestly think it's also probably going to end up encouraging me to exercise more. In fact, that's kind of my own North Star metric right now for all things AI, and not just health products either. But I will know that I am succeeding with AI when I'm spending less time in this chair and moving my body more. And when I feel like I can, you know, untether myself from the desk, I'll feel like I'm, you know, I'm really winning. And I think I'm getting kind of close to getting there. I know I have one friend who uses voice mode like nonstop and says he does a ton of it from the gym while he's working out. And I'm like, all right, I wanna be like you. I'm not quite there yet, but I do think it's really interesting to think about both the— the ability to crunch all this data, but then also the freedom, the liberation from the desk to go out and make more exercise records in the first place. I'm really looking forward— those are major things, I think, on the horizon in 2026 that I'm super excited about.

I think people— yeah, just to add on that, I think people should expect that the value of data that they collect on themselves should increase over time as model intelligence increases. So the ability for models, both from a research perspective and a product perspective, to analyze that data and come up with useful insights that could be useful in the future, I expect to increase over time. And so if there was ever a time to get a watch or anything like that, I would say it's now.

So I do at this point basically everything in Triplicate. For my son, my morning routine is when the lab results come back, export them out of the EMR, Drop the PDF into all 3 of the frontier models. Grok has not cracked that top tier. Maybe it should, but it didn't in my initial evaluation, and I haven't gone back. So it's Claude, Gemini, and ChatGPT. And the sort of tasting notes on how the models interact with me is, in short, Gemini is by far— and this is quite surprising for a Google product, but it's like, definitely clear. It is the most inclined to push me to advocate for something. It's also the most confident when it says I don't have something to worry about. And generally, I would say it's very accurate. Sometimes I'm a little uncomfortable with how opinionated it is because I'm like, what if you're wrong? That's a surprising sort of behavioural profile, I would say, for a Google product, especially because everybody historically has said, oh, they're going to be so conservative. They can't They can't build AI products because, you know, they can't live with this sort of risk or whatever. Gemini 3 does not reflect that analysis. I would say ChatGPT is kind of on the other end of the spectrum in terms of it generally gives me, and again, no system prompt. It does have like my memories and stuff, but no intentional attempt by me to shape how it's responding to me. It tends to give me the longest answers, the most information and is kind of the most clinical and neutral in tone. It's sort of a report style. There's like 9 issues, and we kind of go through and address each one and then kind of summarise them all again at the bottom. And then Claude is kind of somewhere in the middle where it's generally much briefer than ChatGPT. It's more like Gemini-length response, but more kind of measured in a more ChatGPT-like way, certainly compared to Gemini's more opinionated persona. What do you think of that? How do you think about sort of the balance between like thoroughness and digestibility, I guess, is probably the main tension I would zero in on there specifically for ChatGPT.

Yeah, I mean, I think it's hard to know what's right and what's wrong. I think one of the things that we've aimed to improve and also measure via Healthbench, but also improve in our recent models is having the— and this is especially true of our most recent models— having the knowledge of when the user is likely a health professional versus a lay user. And tailoring the response accordingly. And part of this is making sure that you're applying a level of detail or technical jargon that makes sense for the level of expertise for the user. And so this is actually a specific thing that we train and evaluate on. There's actually in our open source help bench eval, there's actually a part of it that is focused on this. And so this is something that we've seen improve, especially recently with 5.2 thinking. And so definitely an area that we've been investing in. But overall, I think it's very hard to say what is ideal and what is not ideal. And I actually think it personally, I think it's amazing that if other folks can, other competitors can be in this space and actually push the Overton window along with us, because I think it's hard for any one company to do. And a large percentage of the work that we're trying to do is think about where the models are at today and where is the Overton window of user trust for the models, and can we move the Overton window along in the right ways? And I think additional models, additional products from other competitors, from other companies, from other players in the health ecosystem actually go a long way to helping shift that as well. And I think it would be a hard battle for us to win if it was just us. So I actually think personally, it's amazing that there's other folks here.

Okay, so it is a striking thing to say that We don't need your data. Basically, we've got enough. I am interested in whatever you can tell us about where that data is coming from. This seems like an area where synthetic data— maybe I'm naive here in saying this, but it seems like you really want some real ground truth from actual human medical trajectories, as opposed to— I would feel a lot more comfortable synthesising chats and trying to dial that in via synthetic process that I would feel like you can kind of fully synthesise data all the way to superhuman doctor performance. So interested in kind of where data is coming from. And I have this notion that you may think, again, is like kind of beside the point, given the tricks that you guys have up your sleeves, but I've been imagining a sort of possible new social contract Not just between patients and AI providers, but even kind of patients and the medical regulatory system, where I'm like, I don't think my son is going to end up in this position. I certainly hope not. There's a lot of people who are in a spot where they're like, I would kind of try anything if it had a chance to save my life. And a lot of times they can't even get access to it. And then now we're in a spot where The AIs increasingly are going to know about it. You might not know about it as a person, but now the AIs are going to know about it. They're going to tell you about it. They're going to make a pretty damn compelling argument in a lot of cases that this is probably the best thing out there. And I have been down this path in kind of a contingency planning frame of mind. Everything's gone well for my son so far, but what if there were a relapse? What would we do? And I've been really impressed by what the models have been able to give me there as well. So I imagine this situation where there's just a mounting pressure from patients that are like, look, AI knows about it, you know, and it's telling me this is the right thing to do, and you're telling me I can't have access. That seems like that is gonna be a very hard gate for the establishment to keep for all that much longer. But then the other thing I would love to see kind of on the other side of that trade is, okay, if you are gonna be given access to unproven treatments, then what we as society Even again, more broadly than the AI companies, what we as society want back are your results, because we want to fold this into our general understanding. It seems to me that we are getting to a point where, and this probably is going to be mostly driven by AI, we could envision not necessarily doing away with clinical trials, but we could envision moving beyond clinical trials, doing a lot of n-of-1 things that are just like, this is the best guess that we have for you based on all available information. And then if we can capture the result of that and train on it as humans, as AIs, as society collectively, it seems like there is so much room to learn so much more, to move so much faster, all while delivering outcomes to at least some people who are trying these things that they couldn't otherwise get their hands on. And I really am feeling motivated to advocate for that. I guess there's maybe 2 data questions of, like, how do you not need more run-of-the-mill data? And what sort of data, you know, is that the kind of data that you do still need? And what would you think about sort of an evolution of the social contract? I call that AI and a right to try.

Yeah, that's a super interesting idea. We talked a little bit about the patient-facing problem of data fragmentation, right? And the experience of this as a patient is your data, your healthcare data is just kind of in the ether somewhere. You don't know where it's going and how it's going going. It's governed by HIPAA for the most part. And the result of that is that there are pathways for that data to go places that don't involve your consent. And so the experience for today for a patient is that you both have limited access to your data and limited control over where that data goes. If you think about it actually from the provider perspective, this is also a problem, right? So if I'm a healthcare provider, I want to understand the whole picture of a patient's health. and integrated across modalities as doctors are so expert at doing. How do we improve that experience? Right now, doctors have a lot of trouble actually, like if you go to a new health system, pulling the relevant records from other health systems, from your entire context. You can imagine a future where ChatGPT actually has a lot of context on you and can the doctor be able to pull that data in as well? I think that could be a really cool future. And then the third point that you're talking about is is research, right? So right now, a lot of clinical trials, the majority of clinical trials that fail, they fail prior to conclusion, are failing due to recruitment challenges, right? And so we basically have a failure to recruit the right patients who meet some eligibility criteria. And a lot of this comes down to the data is not in one place for what these patients are and who they are and whether they are eligible. And so that is also a problem that is facing researchers. And it's is hampering our ability to raise the ceiling of human health as well. So you're pointing out a potential future where not only do patients have more access to data, providers are able to access the data of their patients in a way that's unhobbled as long as the patient consents, but also potentially researchers are able to access the data of a patient, again, if a patient consents. I think it would be really cool to imagine a future where patients are able to opt in for an additional consent and are able to kind of experience the benefits of— what did you call it? An AI-enabled—

AI and the right to try.

And the right to try. I think that would be potentially a really cool future. I do think clinical trials and the standard of evidence there, it's battle-tested, and there is something important about that as well. But I'm optimistic about a future where, because data is a little bit more centralized and a little bit less fragmented, we just have a better ability to advance the science. And because patients have access to their own data, they're actually more— and are able to advocate for themselves because of AI— they're actually more able to figure out what clinical trials they may be eligible for or what experimental treatments that may be interesting for them. So that if they do kind of have that active and informed consent that they can do, that our understanding of science can improve, that our models can improve, things like this. I think that would be a really Optimistic future. And my previous comment, by the way, is not that the models have run out of data. I think data is always a helpful thing for models, but more of a thinking about what is the right way to roll this out to society and where can we get the most— what would be the most impactful path? Is the most impactful path to lean into privacy and lowering the activation energy for users, or is it to lean into getting data? And I think here that decision that we made is the right decision is for our users to lean into not using the data for training our foundation models. And I think that's actually what's going to have the most long-term impact. If we think about the arc of improving AI and improving human health with AI is moving the opportunity window along in a way that's trustworthy, that shows that we respect the value of people's privacy and data. And then in the long run, I think things like additional consent and different changes to the contract of how research is done, I think, are really cool things to explore. Yeah.

Just as you were thinking about this, or as you mentioned, additional consent, I did get a packet of 40 pages of paper at the hospital one day, and they clearly really do want to collect this. There's like a person whose job it is there to go around and visit the parents of these kids at the children's hospital and ask, you know, and explain and answer questions. And I was like probably the easiest customer that she'd had in terms of, you know, I was pretty predisposed to sign. So I signed and I initialed in a ton of places and whatever, right? To share whatever data we could share with the general public good. And still, I just got this in the mail from the University of Minnesota that is like, can we have your data? And I'm just like, man, I thought I agreed to this already. And, you know, what is the yield on those things that they're sending out, right? I mean, it's gotta be like quite poor.

poor.

I haven't sent mine back just because I've been busy, and I fully intend to. So, man, those barriers are really tough. Would you think about doing something like— I mean, obviously, much has been made recently about ads coming to ChatGPT, and I think the argument that, hey, we need some way to support this for billions of people, and ads is a proven way to do that, is a Pretty compelling argument. I don't know if it extends to health. Obviously, there's a lot of pharmaceutical advertising in the world today. Would there be such a thing as pharmaceutical advertising in the world of ChatGPT health at any point? And would you consider, like, you can get ChatGPT Pro for free if you'll give us your data? That seems like something that a lot of people probably would opt into in a way that I would think that they would still feel good about. So I don't know, any other sort of trade? Because again, we do want this to go to billions, right? So any other trades you could see OpenAI making to support that reach?

Yeah, so ads aren't coming to ChatGPT Health, and we don't plan for that right now. Again, we think it's really important to kind of create a clear separation between our health impact work and things that could be seen as contributing to other incentives for the company. And so that's kind of the line that we've taken there. I think you're right that access is a really important point for this, and this is why we've made ChatGPT Health free. This was not the default path, providing a reasoning model for free without rate limits to all users. But that is like the chart that we're— the path that we're charting with ChatGPT Health. And that's actually not a thing that's available otherwise in the product. And so we are deliberately optimising as much as we can for access and doing as much as we can. I think there's going to be some limits to how we can do it, but I don't see any trade-offs in the immediate future. Cool.

Yeah, that's admirable. I mean, I think my sense of OpenAI on this dimension is very appreciative. The pains that have been taken to support a free user base of hundreds of millions of people at obviously not insignificant cost, when probably a lot more revenue could have been extracted from those people, is, I think, a pretty admirable thing. So to see that— and I hadn't even heard that there was this plan to go to free without rate limits for health for all. I mean, I've had this idea for a while of universal basic intelligence, which is basically sort of— again, I'm obsessed with ideas of new social contracts. But this is like one version of it that is, you know, a pretty huge needle mover, right? It's going to be an unbelievable needle mover for a huge, huge number of people. That's awesome. You want to talk a little bit more about kind of the medical establishment's response? At the hospital, my experience is still mostly lack of awareness is kind of what I see. Among the providers, as I've built up rapport with certain people, I tell them, you know, I'm consulting with AI on this, and they're kind of usually okay with that. They're not like hostile to it. They're sometimes sceptical. I have had a couple interactions where they're like, tell me what it said, and I'll tell them, and then they'll be like, oh, okay, pretty good. They kind of— the doubt can turn around pretty quickly when you get the right answer. There is, you know, mention certainly of open evidence. There is Broadly, though, I think just kind of a lack of awareness of how good the systems have become. So my guess would be that sort of awareness is probably the biggest concern that you have. I think in a lot of professions, we should expect to see closing of guild ranks and raising of barriers to entry. And the cynic in me thinks maybe that'll happen in medicine, but the idealist is like, maybe not. Maybe this is a chance to live up to the actual mission of the profession and do the right thing. And maybe it's also the case that people are just so overworked in medicine that they would be happy to take whatever help they can get. But how would you characterize the broad reaction from doctors writ large?

I think broadly, I think you're right to point out that the Overton window shift for consumers has actually been faster than the Overton window shift for doctors. And so already in the last year, the rate of adoption of ChatGPT for health, even before we launched the recent products, was incredible. And it's been one of the most amazing things to see. Again, over 200 million people a week using ChatGPT for health and wellness questions. I think on the doctor side, you do see a rapid increase in adoption, but it's not quite as fast. And so I would expect there that, for that to come with a couple things. One is kind of like interesting interactions between patients and doctors who are maybe at different timescales or at different points in their journey of adopting AI. It sounds like you've had a couple of them. And I think it's interesting because I think a lot of the— when I talk to physicians, a lot of them first hear about AI through patients who are using AI rather than AI that is actually built specifically for them, which is one of the things that we're doing with ChatGPT for healthcare, which we can talk about. That's super interesting. And I think one of the things that we've learned over time is that the best way to shift the Overton window. Like we've done over my time here at OpenAI and also at Google, I've done a bunch of work on studying things, doing real-world studies, things like this. I think the thing that I found most effective for shifting people's opinions about AI and health, especially as the models have become more capable and safe, has been just putting the technology in people's hands. And so that's actually what we've been doing on the ChatGPT for healthcare side. So ChatGPT for healthcare, was announced the day after ChatGPT Health and is actually like more of an industry-facing announcement. And so what this is, is basically like a version of ChatGPT that is actually purpose-built for the workflows of health professionals and specifically clinicians. And it includes HIPAA compliance, it includes additional features like specific evidence retrieval for medical guidelines and things like this, and additional workflows that are specific to to enterprises and writing workloads that doctors do and things like this. And so we launched that actually with 8 of the leading institutions across the country. And one of the most important things for building trust and credibility in the medical establishment is actually working with these leading partners. And so working with these partners, we've been receiving amazing feedback. We've gotten actually a ton of inbound since that announcement, like more than our team can actually handle. And so I think my hope is that this announcement and also some of the work preceding it on the research side with our study, with Panda Health, we hopefully see a little bit of a wave of adoption, not just among individual clinicians, which is what we're starting to see, but also at the level of health systems and potentially even governments. And so I think we're just at the beginning of that wave. I think that's part of what I meant by we are seeing adoption but not scaled impact yet. And I think scaled impact will start to happen, especially in the medical establishment-facing side, more this year. Yeah.

I wonder if there's a question— I'm sure you thought about this— that's sort of analogous to the privacy question on the patient side, that is like, what is the form factor or sort of mode of rollout or use that is going to be best received by doctors? Because if I'm a patient, I might say, hey, what I want is for the hospital, or even some bigger organisation than that, to just architect some workflows that just grind through everything, right? I want nothing missed. I want every record of mine examined. And I don't really care if that offends a doctor at some point. I just want the best results, right? But I could also then imagine that if you were to do something like that, you might ruffle some feathers, create some immune response, so to speak, from the profession that you would rather avoid. So do you have kind of a— is there a way, similarly with the, or analogously to the sort of privacy thing, where you might take a little bit off the fastball in terms of how much you could create in terms of immediate value in order to make the rollout more acceptable to decision makers so that they hopefully kind of work with you as opposed to in some cases maybe be more inclined to fight you over time?

I think the trade-offs here are actually less than one would expect. You mentioned this possibility of possible protectionism in the industry, and we've actually seen much less of that than one would expect. And I think the reason is A lot of these doctors actually use it for themselves or people they take care of. And when you do do that and you see the value of it and you see it getting incredible things correct and maybe pointing out things that you hadn't thought of and things like this, then that becomes like the easiest kind of conversation. And when we talk to health system executives, you can tell instantly who's used it and who hasn't used it for this use case. And the conversation just becomes incredibly easy when people have used it. What we see today, I think, is not a lack of top-down interest in adopting. We actually see a huge wave of top-down interest in adopting. I think the reality is, though, in healthcare that workflows are fairly entrenched in a lot of ways, and it takes some time to make changes. And this is, again, the kind of thing that we found in our work with Penda Health, where we had to do— in addition to rolling out a cool new tool, we did this work on active change management. Where we actually, like, brought people along, showed them how to use the technology, had sessions where we hosted a bunch of them together and had them learn together about how to use the technology more in the future. And that was a really important part of rolling this out. And that kind of change management, I think, will be really important here as well. So I do think that takes a bit of time. But my expectation is that this will be one of the faster rollouts of software in healthcare history. And I think that's already been happening with people's usage of AI. And I think that's just at the beginning. So I wouldn't expect— I don't think it'll take years and years for people to adopt AI more and more in clinical workflows. One of our goals for this year is for AI-assisted care to become more part of the norm of care. And I think by end of year, that'll, I think, be the case, we hope. But I don't think it'll happen over weeks, which is something that us AI innovators are used to, I think.

Let's talk about the connection between health and AI safety more broadly. That, I think, is super interesting. I remember the classic question from Ilya once upon a time, how do we teach AI to love humanity? I understand that notion is not entirely gone from OpenAI and is still part of what you're thinking about. So love to hear more about that.

Yeah, absolutely. I think it starts with kind of the foundations of the work that we've been doing on health at OpenAI and how we started it. So I have a bit of a background as a researcher who cares about safety and has worked on safety research in the past. And part of the motivation for coming to OpenAI and to work on health for me was thinking about the setting as a place that can provide a kind of concrete grounding for technical work on safety and alignment. And I had a feeling, and this was about 2 years ago, I had a feeling back then that a lot of the most ambitious work, the most medium and long-term work on safety and alignment was going on in kind of toy settings or going on with math problems or things like this. And it felt like if there was only a setting where the problems that people were working on were well motivated, that provided more concrete feedback loops to researchers and more short-term and incentives for the research, that the research could happen better. And so that was kind of part of the thesis for our approach for work at OpenAI. And we've done a few kinds of research, and I talked about one of them, which is our kind of work on calibration. Another problem that we've really thought a lot about is thinking about scalable oversight. So how do we supervise AI systems that are potentially more capable than us in certain ways? And this is a problem that we've actually had for some time in our work with physicians. Right. So in many ways, AI doesn't match your expectations of a doctor and the ability to kind of integrate across a bunch of different modalities and things like this. But in many specific narrow ways, models can even outperform physicians. And in those specific narrow ways, when we evaluate models with physicians or have physician signal be part of the training signal for training models, we have to invest in research that is around this problem of how do you supervise systems that are potentially more capable than you, which is one version of the problem of scalable oversight. And so this is a problem that goes beyond health and is actually, I think, a really important problem for thinking about how we think about AI alignment. I think one of the more important problems there, but our work in health has given it a bunch of concrete grounding because we do have models that in some ways are more capable. And so that's been a lot of our focus, has been thinking about the right ways to approach that problem and make that better. And again, that has kind of been some of the work that we've done on evals, some of the ways that we've pushed the high compute RL paradigm in the setting so that we can learn again from the aggregate of the opinions of experts and things like this. And that's been an important part of our motivation. But broadly, I think a lot of where this has been heading for us has been thinking about how do we kind of extract out the right personas or characters from the model? Because if the models become more and more capable over time, they have more and more inputs. Let's say they have more and more context that they're getting about a patient's medical record because users are more proactively uploading them, or the activation energy is lower via product, and they have more ability to take output actions, whether that's telling you things or outputting a note that you can give to your doctor or other things that they can do. Probably the most important thing in the very long term is, are the models the kind of models that would do the right thing for the patient or for the user or for the for the clinician or for the researcher. And so that's the kind of thing that we've been investing in a bunch is like, how do we think about extracting the right kinds of personas for models? How do we do so in a way that's surgical, that gets the best aspects of the personas and gets less of the ones, the parts that we don't like? So, for example, avoiding a bias towards being overconservative in challenging situations involving unclear medical consensus, but also navigating the uncertainty super well. And so that work, which is kind of closer to persona or character or soul in its form, actually we're grounding a lot in work in health, which has been a really exciting advance. And I hope we'll have more to share on that in the future.

Can you talk a little bit more about scalable oversight? I mean, this was, I think, at one point kind of the plan, right? The Superalignment team, I think, was sort of premised on this idea of, and maybe other ideas too, but Certainly my understanding was scalable oversight was going to be a big part of it. I think one of the things that stood out was, like, the strong student sometimes was just, like, ignoring or sort of overriding the instruction of the weak teacher. And it was like, we're the weak teacher in this situation, right? So how are we supposed to feel about the already emerging trend as of probably at least 2 years ago now, that at times the strong student just decides that it should ignore or exercise its judgement, despite the fact that it may contradict what the— we, again, humans in the analogy, are the weak teacher. Have there been paradigmatic advances in that that give you confidence that this is going to really work? Or is it kind of the usual understanding I have today from OpenAI broadly around safety is like, well, it's going to be sort of a defence-in-depth strategy. Everything will work a little bit, we'll kind of gradually chip away at the problem, and hopefully with enough layers of defence, it'll be okay. Do you have more ambitious ideas about what scalable oversight can achieve still in your mind than that?

Well, I think my thinking of how scalable oversight will proceed has kind of changed a little bit over time. I think of the problem as having 2 parts. One is, I call it radar scaling, which is how do you think about the right ways to elicit the opinions and values from people or experts or whatever? And that's an example of something that we've been investing in, in health. And so then you can imagine schemes where you have AI be part of that loop and help improve the ability for humans to critique, for example, AI outputs. So that's like one area of work that the company has been investing in. The second area of work is given some— I think the framing for this has become more expansive over time, and sometimes we don't use the word scalable oversight to refer to it anymore. And I think the same is true at the labs. Given some idea of what the values are, whether they're elicited using radar scaling or not, how do you spend a lot of compute on training the models to have those values? And so I call that internally value oversight. So there's radar scaling and there's value oversight. And so the second problem I think is also a problem where even though we don't use the word scalable oversight, I think things have advanced quite a bit over time. So one example of this, which again doesn't use the word scalable oversight, is the work that people are doing on specs and constitutions and measuring and improving models to adhere to these. This is one way of saying we have values that we care about and we want the model to be very persistent in improving them. And I think if you check various system cards or things like this, you'll actually see that the models have gotten much better at doing this. And I think people have been finding increasing generalization in training models to have certain personas or characters or things like this for whether they comply to certain safety requirements or specs or constitutions or things like this. But what I'd say is basically I think the research has been actually advancing, but in a way that actually looks different from what it did before. It looks a little bit more like humdrum post-training than people imagined it would look like. And I think this is a good thing. I think this is like a little bit more grounded in how systems will look and how we will train them and things like this. I don't think we've solved all the research problems, but I think sometimes we're working on the research problems without referring to them in the same way as we used to.

Would you say the core of the reason to think that this will work is that a somewhat less capable model with a super big inference budget can be expected to catch? If we're thinking of a deceptive sort of failure mode, the smarter model could, even if it's more capable per token, the somewhat less capable model with just a much bigger token budget, it's kind of— I mean, this idea has been out there for a while, but it's really sparked when you said spend lots of compute to make it work, just really giving that thing a lot of time to think. It can be enough as long as the delta isn't so bad that it can detect anything that's going wrong in the more capable model. So as long as you've got kind of a small enough delta and a sufficiently aligned current model, then you can, in theory, continue to bootstrap your way into ever more capable models without losing the alignment. Would you say that's kind of the core of the idea?

I think that's part of it. I think another part of this is that the task of discrimination or critique seems to be easier than the task of generating good outputs. And so when we study the performance of monitors, a given model can monitor itself, especially given privileged information like the chain of thought, pretty well and can actually, which is a little bit surprising on its face. But then if you think about the fact that discrimination has been better than generation, and this is kind of underlying a lot of work in things like RL-AIF and constitutional AI and things like this, then it's not so surprising. I think that's another part of it. But I don't think we have a full understanding of how this will scale, especially when we're thinking about the regime of having trusted but less capable models aligning a more capable model. I think so far I've mainly been talking about if we have the values and some format, whether it's from humans or models, then how do we instil those into models in a way that is trustworthy? I think that's a part of the problem, but I don't think the whole problem is solved for sure. Yeah.

How do you think we're doing on safety as a whole? I mean, I have been an Eliezer reader since 2007, and I actually read a lot more of his stuff way back in the day than I have more recently. I would say, broadly speaking, it's like, Hey, this has in some sense gone amazingly well relative to baseline expectations, right? We do have models that undeniably have a pretty good sense of human values, and that's just manifestly obvious on a day-to-day basis. That was, I think, considered to be a very unlikely outcome years ago. And if you were to teleport back to 2007 and drop a post on Overcoming Bias indicating as much, It would be shocking or it would be considered laughable that this would be accomplished in this way. And yet we do still kind of see these, I guess, my mental model of the sort of seesaw that it seems like we're on is like every generation of new models has new capabilities generally, and then it also seems to have usually some new emergent problem. And it was, whether it's like deception or now we've got eval awareness has become front and centre, and we sort of are like, okay, that's a big problem. The next generation gets more powerful. We kind of tamp down that last problem. It doesn't go to zero, but it's at least reduced. But it seems like we're sort of headed toward this strange world where if you sort of extrapolate both the meter trend and the kind of whatever, 2/3 to order of magnitude reduction in bad behaviours that seems to happen at least once a given bad behaviour is recognised and addressed from one generation to the next. If you kind of extrapolate that out to 2028 or whatever, you're like, okay, I can imagine a model now that can do a month's worth of human work or a couple of months' worth of human work, but it also maybe has, for any given run, like a 1 in 1,000 or a 1 in 10,000 or 1 in 100,000 chance of actively screwing me over in some super bizarre way. And that just seems like a really weird thing to contemplate. It's a really weird world to live in. But if we believe in straight lines on log graphs, that seems to be kind of where we're going, right? So do you think that's where we are going? And if not, maybe you have a different mental model of what the sort of alignment versus emergent problems balance would be in a couple of years' time?

Yeah, I think the future world will definitely be pretty weird. So it's very hard for me to predict what will happen. I do think models will become more capable. I do think they will be able to do things over longer time horizons. I think that's very important. I do think also we've been pleasantly surprised about the extent of safety generalization or alignment generalization, which is underlying the trend that you're pointing out of a lot of various safety benchmarks or failure modes decreasing over time. And so that's been a relatively pleasant development because I think if you think about it, that was the case for— it was the case during— we've had 2 major kind of scaling laws for deep learning. One is kind of the pre-training scaling law. What we found is that when we scaled up pre-training, we had models that were relatively good world models that could be relatively easily tuned via SFT or small amounts of supervised learning to exhibit a certain persona and extract that persona and things like this and be helpful, useful assistants. And that's a lot of what the early work on instruction tuning and things like this did, and also what the early work on Med-PaLM did for the health setting, which is really great. And I think we saw a lot of generalization there. I think there was a question for the reasoning setting whether that generalization would hold or we would see similar generalization. And so, so far, my sense is At scale, we do see it. At smaller scales, we saw less of it. And so that's a promising development, which is that if you are able to figure out the right ways and patterns to get these models to be broadly beneficial and not harmful, that even if they're put in settings where they're doing things that you didn't foresee, doing a month's worth of work in days or things like this, that they can continue to generalize and be safe in those settings. And that those safety curves will continue to go down. That said, I think you're right in pointing out there's a rapid increase in capabilities and surface in which these models will be used, and also a rapid decrease in safety failures that we're seeing over time. It's a little bit hard to predict how both those curves will net out. I do think it's really important, and this is why I care so much about the parts of our mission that are about proactively making the tangible benefits happen and also about working on mitigating safety risks, that we really stay ahead of these curves and think about the right ways to shape them in the right direction.

What does a Move 37 look like in health? And are we going to see that in the current paradigm, or would it take a sort of deep integration of modalities like we were kind of touching on earlier? Or another idea that I'm kind of enamoured with is this idea of maybe a different kind of training objective, thinking about instead of getting right answers to questions, more about at a fundamental level predicting what the state of health is going to be for a patient in a way where you could even begin to do in silico experiments, right? Like, hey, I've got this whole profile of this patient. What if I change this? variable? You know, what would their health look like in that case? That seems like it might be a pretty different paradigm from kind of like, read the whole internet and make sure you give me the right answer. What do you think? Touch on any of those that you want, but maybe also just take the opportunity to zoom out and give us a sense of like, what are your big visions and ambitions? And what do you think people can kind of expect as you guys continue to do your thing over the next year plus?

Yeah, I love the move 37 framing. Just to explain the reference, this is a reference to the famous move 37 during a game between AlphaGo, the AI system playing Go, and Lee Sedol during one of their now famous games. And the interesting thing about this move was that it was a move that everybody agreed humans would not have made, but in hindsight was brilliant and was key to winning the game. And so I think your question is kind of like, can we imagine a world where models are able to do something and maybe make some kind of interesting prediction That a human probably would not have been able to make, but was again brilliant and impactful in hindsight. My view is that this is not too far away. I think in many ways, many people report to me that they saw many doctors for their case and only after talking to ChatGPT was ChatGPT able to flag the thing that they then shared with their doctor and then they were able to come to a diagnosis together and things like this. I think that seems seems to happen somewhat routinely. And whether it rises to the level of a Move 37 or not, I think, is a matter of taste depending on the case. I think what I'd say is that I think you should expect world models in health to improve. And what I mean by world models is we should expect our models to have a much better understanding over time of us, our health, our trajectories of our health, and how that intersects with our biology. I think that's going to be really key to thinking about what this Move 37 could look like. You mentioned this idea of simulations of people and simulating things in silico. I think full simulations are obviously very expensive and difficult, and I think isn't a thing that models are most designed to do in their current form. So I would say that there probably would look a little bit different from the current kinds of training that people are doing. But if it's more along the lines of given a lot of context about a user, predict something interesting about them or predict the results of some intervention. I think this is something that models first are already getting better at, and second could get a lot better at over time. And I think that would be potentially quite impactful. I think we'll start to reach a point where there'll be more and more clear demonstrations of this. I think you're starting to see interesting and increasingly clear demonstrations of models doing interesting science and math in the public. I think you'll see a little bit more in this space as well. But I do think it'll take a little bit of time. My view on paradigms is that you can get a lot out of extending and tweaking the current paradigms. You can in fact view everything that has happened so far as just one paradigm. But between pre-training and scaling reasoning-style training, I think you can get quite a lot of juice. I think even integrating additional modalities and things like this, I think doesn't require too much additional changes or tweaks? I think broadly, our team's mission today is do whatever it takes to ensure AGI is beneficial on human health for all of humanity. And so we see that happening through 3 channels. The first is helping consumers understand and navigate their health, which is already happening through ChatGPT Health. The second is empowering the health system and thinking about the ways in which AI-assisted care can become part of the standard of care and it can reduce the extent that clinicians are bogged down by paperwork and are spending more time thinking about the— seeing patients and thinking about the problems that matter most for improving care. The final thing is really pushing up the ceiling of research. And I found your vision pretty inspiring. And so what I would say is, like, I think you can expect, as a lot of the core problems in healthcare that we've talked about, the fragmentation of data, fragmentation of patient experience, to improve. As those happen, you should expect an acceleration of the application of intelligence to that data. And I think biology and health is one of the areas in which marginal gains in intelligence have the most obvious value in solving more and more problems for humanity. There are many examples of previous breakthroughs in biology where there was nothing stopping that breakthrough from happening 5 or 10 years earlier except just more human ingenuity applied to it. that there's no physical blocker. And so when that's the case, then I think you can assume that long-running models, long-running agents connected to the right data can do really incredible things. And I think the hope is that in addition to our existing work, which has really been focused on how do we raise the floor of human health, we start to raise the ceiling of human health as well.

Yeah, it's going to be not just the AI doctor, but also the AI biomedical research scientist as well. Wow. Okay. It's an exciting time to be alive. I'm increasingly saying that at the end of these conversations these days. So, yeah. Anything else you want to make sure people are aware of that I didn't ask you about?

Like I mentioned earlier, I think the right way to think about our work for health at OpenAI is really operating in 3 phases. One is really laying the foundation, and a lot of that is focused on our work on safety. And so examples of this are like our work on Healthbench, where again, we have this kind of evaluation which you can run offline of large language model performance and safety and health. We had this study of the first AI clinical copilot with Panda Health. We had a bunch of model improvements over the last year, and all of these kind of lay the foundation for the work that we're doing today. The second thing is this kind of rise in adoption, which we've been seeing. We've been seeing an incredible rate of individual users, especially patients adopting technology, and we've seen hundreds of millions of people asking health questions a week, which has been in rapid growth since last year. The final thing is really the future of scaling the impact of this work. And that's where really our work on ChatGPT Health, which is this kind of consumer-facing product, which enables you to connect your health data with additional privacy protections. And also our work on ChatGPT for Healthcare, which is again facing health systems and really enabling health professionals to use AI as a copilot in their workflows. kind of come in. And so between all these things, I'm really excited about the work that we're going to be doing to scale the impact of this work in the next year. And I'm looking forward to what's to come.

Karan Singhal, Head of Health at OpenAI. Thank you legitimately for all, all your hard work. It really is incredibly valuable, incredibly impactful. And that's obviously just going to continue to grow exponentially along with so many things in the AI space. And thank you for being part of the Cognitive Revolution.

Thanks for having me. Every generation try feel be depend. Every mother praying that her child remain. But something in the wiring of the world, we a copper river of the known start feel overflow the cup. Move 20 sas and the ghost in a discount. He want no doubt. Quarter billion people asking every week, finding words the system had but doctors couldn't speak. Chain of thought in English like a river running clean. Alignment wasn't programmed, it's the realest thing I've seen. Kenya Clinic Sunrise, Vanda Health Alive, free intelligence, no gatekeep, no jive. Doctor in a machine. Raise it up. Doctor in a machine.

If you're finding value in the show, we'd appreciate it if you'd take a moment to share with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course, we always welcome your feedback, guest and topic suggestions, and sponsorship inquiries, either via our website cognitive revolution dot ai or by DMing me. on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts, which is now part of a16z, where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at aipodcast.ing. And thank you to everyone who listens for being part of the cognitive revolution.

Thank you.