Vinod Khosla + Lisa Weiner Intrator on Deep Tech Health | ApplySci Silicon Valley

Hi everybody.

Hi everybody. So Vinod is— actually, I've been talking about you. I talked about you in my opening remarks. You weren't here this morning, but I noted that 8 years ago you said that AI was going to dominate medicine. And it was controversial then, and now it was visionary, actually. So first of all—

Or a lucky guess.

So Vinod is very modest, obviously, and very visionary and very open. So he likes to structure this talk by opening it to questions from the beginning. So I would like to start actually talking about AI and maybe talking about some of the things that Nathan and Mikael brought up. And then we're really open, so just feel free to jump in and ask questions. So is that OK with you?

Sure.

So my first— and first of all, thank you for being here. It's the fourth time that you've done this. I'm starting to take you for granted. It's a good audience. So it seems from some of the talks today that there appear to be two approaches to AI in healthcare. One, as Nathan said and as Mikael said, is taking kind of off-the-shelf AI and applying it to massive data sets. For example, what Amit Etkin published yesterday about depression and biomarkers. Another is what Mary Lou Jepsen, who has been at this conference several times, is doing.

And is really—

We are investors in Mary Lou's.

I know. Yes, that's great. And she should be here today. I don't know where she is. But taking novel miniaturized sensors and doing things that are maybe higher risk, but maybe much higher payoff in changing the world. What do you see as having the greatest impact on society over the next 5 years?

Well, you know, oddly, you mentioned 8 years ago, it was highly controversial.

Say something controversial now.

I'm sure during the course of this conversation, I'll say something controversial. So I'm never lacking for that. But if you look at it logically, wherever there's lots of data, AI plays a large role. And sensors, of course, play into generating lots of data. And it doesn't matter what the data source is. If you're collecting a lot more brain data, then AI has a lot more to work with, and you're going to see results that are far beyond what any of us expect. And one of the odd things about AI we've discovered is more than almost anybody expected, even 2 years ago, the scale of the compute— which one would expect scales sublinearly, is super linear. It means the larger network gets, your compute network gets, the more data, more insight you get out of the AI data, which is surprising to most people. So data and sensors are key to this. And it doesn't matter what area of, of AI we are talking about. 8 years ago, everything was controversial. Today, almost nobody expects humans to do better than AI in any image-based sensory domain.

Not almost nobody, nobody.

In fact, the editor of the American Journal of Medical Association Journal I think it was the JAMA journal said last year that they're no longer accepting papers proving humans are better than— AI is better than humans at any kind of image analysis, just period, which is pretty stunning compared to where people were 8 years ago.

And actually stunning compared to where you and I had this conversation 4 years ago. People in the audience were still like, oh, well.

Yes, people were very skeptical. I actually recently looked at the 100-page document PDF I'd done about 5 years ago, and it didn't need that much reworking. So I didn't rework it, and I left it as is, which is pretty telling. I think it's pretty clear on some axes I've been more conservative. But take an area like drug discovery. Vinod Khosla. Not an area you'd expect. You know, if you sort of take 400 million compounds possible and say, I wanna, in high throughput screening, screen for 4 million of them, some expert selects them. It's very, very clear now for anybody working in that field, if you have the right data, then you can do easily 3 orders of magnitude. I just saw a project that screened 11 billion molecules, many of them not even synthesizable, for effectiveness on a particular receptor. So whether it's imaging or that, it's very clear that's happening. Now let's look at the other areas. If we have better EEG data or fMRI data or highly complex data, or most activity data of your wearable sensors, the Apple Watch, you're going to get a lot more insight out of it. Now take the coronavirus. How would you monitor that? I suspect— and this is just speculation, not any proof— that when you get an infection, you don't know, feel it for 14 years, but your body starts responding. If we have enough sensor data, whether it's your Apple Watch data or other biomarkers that we've heard about at this conference, almost certainly is going to be the only way to detect 2 weeks in advance, or in the case of flu, 2 days in advance, that your body is starting to respond and start to fight that infection long before you feel it. That's a data problem, a sensor problem, and an AI problem. Recently, we took one of the largest drugs on the— in the world is anti-TNF inhibitors like Humira, $15 billion. For those of you who don't know the drug, only about half the patients respond to the drug. And the drug costs about $50,000 a year in the US per patient. So it'd be extremely valuable if you could tell who is going to respond to this or not. A guy who best be described as a mathematician, Barbasi, for those of you who know him at Northeastern, did a network theory-based analysis. He has no background in biology and can predict with something like 90% sensitivity and specificity based on network theory, which is lots of biomarker data. That we can predict who will respond to this very expensive drug. Bad for Humira, it'll cut out half their sales. Great for insurers, it'll save them a lot of money, and great for patients because they won't wait a year to find out if it works or not. So those are examples. So that leads me to one other thing which is important to all of this audience. One of the fundamental things we have to discard in mad medicine is the notion that humans have to be able to look at the data we collect. You know, in 128 channels of EEG, you're going to see stuff that humans can't make sense of, but is not hard for an AI to. If you look at a company like Somalogic, they measure 5,000 proteins in your body. No human physician could look at that body. But it's not surprising if there's a few thousand metabolic pathways in the body. This complex of data represents how the system is behaving, and that's what we want to know. Almost certainly, if you're going to get Alzheimer's, your system's starting to change 20 years in advance of that. Same is true of cardiac disease. So we need to go to data-driven medicine and collect data, not twice as much data, but 1,000 times more data. I have a funny joke. Around the time I was talking about data in medicine, Stanford was planning still to build their new hospital. And the CEO brought the design team from IDEO over to my office and said, what should we be thinking about in designing the new hospital? It was a multi-billion dollar project. They were just looking for opinions. And I said, you, and I looked at what they were doing and I said, you're off by a factor of 1,000 in how much data you think you'll have per patient. And I'll tell you, recently I was talking to the same person who's no longer at Stanford and I said, I was wrong. By a factor of 1,000 in the wrong direction. And that's the scale of these changes that are going on and the opportunity in front of us.

So what is going to be the role of doctors and hospitals? You know, we're saying AI can read scans and sensors can detect Alzheimer's disease so early. What are doctors going to do? Well, talk about the human aspect of medicine.

First, you say, what's the right treatment for the patient. That's where you have to start. Our job is to provide great care for patients, not to provide great employment for doctors as our first task. There's a secondary task we can talk about. So if we have a company that can do thousands of biomarkers, let me say, and tens of thousands of unidentified but constant biomarkers— that means biomarkers you can't name, they'd be just features in a chart. For under $100, for less than the cost of taking your normal blood test at Quest, for example. That's incredibly valuable in characterizing disease. And Freenome just published a study saying that pretty good sensitivity and specificity in detecting colon cancer using very large-scale biomarkers. So I think we have to realize that's how disease should be diagnosed. We have a company called Inflamatics that looks at a pattern of gene expression to detect sepsis well ahead of any other way to detect sepsis. If you're late by a day in detecting it, your mortality rate goes through the roof. So— What should we be doing? Caring about patients. Having said that, at least for the foreseeable future, there's a human element of care. And I think doctors will do that. Now, it leads to the question, do you select the highest IQ doctors? I guarantee you the people who get into Stanford Medical School or Harvard Medical School are the highest IQ but not the highest EQ. So if I'm looking for the human element of care, which is very important, then I actually think nurses do better than doctors. So who's best equipped to provide the human element of medical care? It is a role for humans, for sure. For the foreseeable future.

I can ask questions all day. Are there any questions from the audience before I continue? I see a hand, a few.

Hi.

I'm curious about your thoughts of the future of mental health care. So in psychiatric illnesses, there's enormous heterogeneity. In the manifestation of illness, enormous societal costs, unlikely to have sensors that consistently, cross-cutting populations will detect illness. And so what's the vision?

You say unlikely?

For sensors. And so if you take a— well, this is debatable. But for illnesses that are at baseline ill-defined and probably don't fall neatly into categories, How can these approaches be applied well to reconfiguration of mental health care?

So the first thing to understand is we know very little about the brain. So if you read the DSM manual— and I remember when DSM-5 came out, Scientific American actually did an assessment of the DSM manual. And the exact quote I remember is looking at two diseases. I think it was bipolar depression— bipolar and manic depressions. The DSM-5 said something like kappas of 0.2 are acceptable. And Scientific American called them two pathetic kappas. Scientific American is not the National Enquirer on these topics. My point is we don't have data to know what diagnosis— diagnosis you get is the psychiatrist you get, not what the disease you have, mostly in mental health.

And it's a psychiatrist who sees you once a week or a month or one time.

Yeah. So we don't even know the very basics of— so an SSRI is a solution for almost everything. But we don't know whether— and if you look at the new theories around inflammation in the brain, that's a very different set of causal conditions than serotonin deficiency or something, or anxiety as a GABA glutamate imbalance. If you can't measure GABA glutamate, how do you know whether there's a imbalance or not? And you give the same drug. So my point is the following. We have nowhere near the right amount of data, whether it is about brain activity EEGs, fMRIs, and I think we need not 10 times more, not 100 times more, we need 1,000 or 1 million times more. And so there's a large opportunity in working on those sensors. And so we can understand the brain, which is hopefully the most important part of the body. Without that, we're not going to have good solutions. And today, with psychiatry, psychology, other areas, we do the best we can. With relatively imperfect tools. It's not the psychiatrist's fault. Their set of tools is fairly limited. A paper and pencil questionnaire for a PHQ-9 Square is sort of pathetic. Now we're starting to do much better. You heard about EEG, voice as a biomarker in mental health is becoming much more common. Or one can now prove that one can get a relatively good PHQ score from a 30-second speech sample. So things like that will increase the amount of data. Don't just think very traditional data. And I think we'll do much better at diagnosis. But I don't think we improve diagnosis without first improving measurement. We have a company called Neurotrack that actually looks at your eye movement when playing a game. Turns out Alzheimer's brains respond to novelty differently and reflected in eye motion than, than normal brains. And so by watching somebody's eye movement during a test, you can actually predict the level of severity of Alzheimer's or even if they have it. So we need measurement, I would say, in general, in many, many dimensions, whether it's sweet, speech, eye, fMRI, EEG, MEG, I can go on, but way more. And I'm encouraged, I think we'll sit here 5 years from now saying we have way more data than we imagined 5 years ago.

Are you committing to sitting with me 5 years from now?

We'll see.

We'll see. And I'm also encouraged by companies like Mindstrong Health who are using the phone to capture mental health data. And tomorrow we have a talk by Dror Ben-Ziv from the University— oh, hi, Dror.

What's the talk about?

So technology-based mental health. And one of the— when George said to me, why do you want me to speak at your conference? And one of the things that really stood out to me was using the phone to discreetly treat mental health, for example, as he does in the West Bank where there's a population of PTSD, and in Ghana. So I didn't recognize you.

Look, we don't even know whether PTSD is a mental health condition or not. There's a research project that I've funded to see if it's a cell danger response condition. And that the mental health piece is a consequence of cell danger response to extreme stress in some situation. The cell danger response lasts for years. Don't know. So we can't answer these. We can have opinions, but opinions don't matter. We need to— to get the data to prove it out one way or another. So I am pretty encouraged that we are paying attention to it. In mental health, I just read a paper speculating that there's not one gene— as you know, the efforts at identifying genes for particular conditions has been relatively unsuccessful. There is no gene you'd say contributes more than 1 or 2% to a particular condition like ADHD or depression. But the research seems to be indicating that a pattern of genes, hundreds of genes, each contributing in some way to propensity to have mental health issues is likely what the genetic basis is. That means your brain's preconditioned. And so if one sibling has, say, depression, the other sibling is more likely to have other conditions we think are unrelated to depression. That's pretty interesting data. That, together with some nonlinear dynamical models, for those of you who are engineers, of brain activity and patterns of firing. I suspect we know the direction to look in, not what the answer is yet. Sorry for a long answer.

There was another question from Amos Niv.

Hi. Coming back to the original question about the future of AI in medicine, and we talked a little bit about this today, today, previously. It seems like most of the efforts are on data collection, sensor data collection for prediction of disorders. You also mentioned early detection as important. My question is, how do you see AI for treatment optimization? Obviously, I'm asking this because I'm involved in that era or in the direction. Our startup made a switch and focusing rather on data collection for diagnostics rather than— we have a wearable therapeutic that actually delivers neuromodulation. But we're looking at AI for treatment optimization, especially with the era of home care therapy where we will be collecting a lot of data. We can get feedback on—

outcomes?

Well, there's two ways. We can do a blind search or we can do a directed search, right? And we are sort of doing blind search because we have no ways to do directed search, and I'll explain what I mean by that, of what the solution is. So whether it's VR therapy or simple thing like exercise helps with depression or exercise help slow progression of Alzheimer's? We don't know the answer to that question. Why? Because we can't quantify the degree of Alzheimer's, the degree of depression. Without that measurement system, it's a blind search. We try 16 things, and intuitively, some things work. If we can measure, we can close the loop. Is there a certain pattern of VR activity reinforce certain brain circuits that then lead to certain consequences. Unless we have those measurement systems, I don't think we'll make progress rapidly. And directed search with feedback loops or closed loops is what we need for rapid progress, and we are not there yet.

To follow up to that question, where do you— how do you see the path of all these AI devices and like NeuroTrack or VR devices to get into clinical practice? So for example, I mean, I'm a practicing clinical neurologist at Stanford, and most commonly, like NeuroTrack, they came to me or other companies like, um, VR company came to me and they're like, can you implement this? So where do you see the clinicians going to their administrator and ask, I need this, and what basis you think it will happen?

So that's a very important question and the answer is highly variable, but let me define two really different paths. One is what you'd expect. Prove clinical efficacy, do the trials, randomized controlled trials, and then you have something you can— I can come to you and say, here's the results. And then there's 10,000 of you in the US or 100,000. It spreads slowly. So if I have a great breakthrough today, it's 10 to 15 years before 80% of the physicians accept it as practice. Probably 5 to 7 years before 5% of the— so the penetration rate, even after you prove efficacy, is very slow. What I'm encouraged by is there are alternative paths that don't deal with the healthcare system to do this. So take Ginger Ayer. Mental health was a question. They go to employers and say, offer this as a benefit, and you know within 6 months whether your employees love it and value it as a benefit. You can measure the level of participation in this benefit. And it doesn't matter what the degree of clinical validation is. You have a different validation. Do your employees love it? Of course, you don't want to do harm. And you have to do internal tracking and studies. And if you increase the number of suicides after you offer this service, that's a problem. So I think— Some of these companies, whether Livongo is doing diabetes or Hello Heart's doing hypertension or Ginger.io is doing mental health, we've seen companies in almost all these specialty areas. We have a great company in physical therapy post-surgery. Complete AI-driven physical therapy, right? Monitored by a physical therapist. But you use 1/5 the number of physical therapists. Guess what happens? People can do their own appointment at 8 o'clock at night. Of their 1,000 patients who were active last Christmas, shockingly, 71% of their patients did physical therapy on Christmas Day. First, even if every physical therapist was open, you'd never see that kind of compliance. And of course, they're not open. And they're not open after 5 o'clock. And I can go on. They get average session times between 5% to 7%— or 7 times a week of physical therapy, depending upon the population. And these are normal employees of companies that just offer this as an alternative to physical therapy. So I think making it better, trying things, and then iteratively with appropriate safety guides, trying these things and improving them. If you're going to improve physical therapy through the regular means, it's going to take you forever. Not only is the AI doing physical therapy, it's collecting an incredible amount of data. And outcomes because they measure outcomes. For physical therapy, there's very clear outcomes. What angle does your knee bend to? And when you have quantitative measures, it becomes really easy. In mental health, there is a PHQ-9 score, poor as it may be, but it's accepted as the standard of care. So you can take these. In hypertension, it's BP reduction. And behavioral health, we are finding, you can do 20 millimeters through digital— of BP reduction through digital therapy only. So across all these, if you can measure, which is why I was focusing so much on measurement, you can actually develop these much faster ways to evolve these therapies than going through trying to convince Humana in a 5-year pilot that this is going to work. Both are valuable and sometimes if you're doing heart surgery you can't iteratively try it. And there are situations, but in many cases you can. And these alternative health plans, especially direct-to-employer, are creating channels with clear measurement and efficacy measurements. And customer satisfaction. And those are really promising channels. The other is the uninsured population of people who have no access to care. Any care with appropriate, again, safety and ethical guidelines helps you develop those. You know, I grew up in India for the first 20 years of my life. I'd never even heard of a psychiatrist. Never heard anybody— I never met anybody who'd ever seen a psychiatrist.

Or at least told you they did.

But you never heard of anybody being a psychiatrist. I used to know the number of psychiatrists in the country for a billion people. It was like ridiculous. Li lo. So there are environments in which the next best alternative poor or rich, is something that's better than nothing. And Michael Bloomberg is very interesting. He's running for president now. But one of the—

Okay.

One of the projects he was doing is teaching high school graduates in Tanzania to do C-sections on women. Because if you needed a C-section, you had a death sentence in almost all cases. If you look at the number of physicians who could do C-sections versus the population, the ratio was off like more than 10 to 1, probably 100 to 1 of what it is in the US. So what is better, death or somebody who's watched 100 such operations do an operation? So it's not perfect. But there are wipes. And so there's ways to be opportunistic. And entrepreneurs are very good at hacking the system to find ways to test things rapidly. Fundamentally, I believe for rapid progress, we need rapid learning loops, which means you have to do things, learn things, and do them again a little bit better. And once you get on this exponential learning curve, Things move much faster. But through the traditional healthcare system, it'd be very, very hard.

May I ask another question?

Oh.

Quick question and fairly general. Speaking more of like neurotechnology or ancient neurotechnology, where do psychedelics play a role in this space?

Ooh.

How many people have read the book How to Change Your Mind? Very few. Everybody should read it. It's actually a pretty interesting— as somebody who figured I'd never try psychedelics, I actually thought the best thing to do was— next best thing is read this book. Clearly, because the area has been taboo for reputable research, we've not understood it as well as we should. And I think we are starting to see a change in whether it's psychedelics or microdosing of psychedelics to study the phenomena. So could I tell you anything? I don't have any opinions. I do think fringe areas are always worth studying if there's a scientific approach that's possible. And I think we are getting to the point where a scientific approach to that is possible. So—

We just had this conference at Harvard Medical School a few months ago, and Brad Ringeisen, who is the new head of the DARPA Biological Technologies Office, gave his first public speech after being appointed. And he acknowledged that while the US Army cannot ever support psychedelics in its treatment of PTSD, he seemed to acknowledge that there is some good science science there.

Well, and even 5 years ago, I do not think anybody reputable would say it's worth studying. And that's a shame. It's a little bit like cold fusion too. It's just taboo to study, which is a shame. And most good progress happens at the edges of the system in unconventional ways, and so if there's a reason— and this is the key— if there's a reasonable scientific methodology applicable to the area we should study.

Hello, Christoph Leuze from Stanford University. I have a question which is probably not as exciting because it's not about the technology, but more about the infrastructure and the regulatory questions. So it's highly sensitive data, but how would the infrastructure need to be so that it leads to the biggest benefit to the patient. And that is also because if you have only one company that, for example, measures biomarker A, another company measuring biomarker B, and so on, how do you combine these? Because at these intersections where you have access to multiple biomarkers, you probably get the best results. How do you put an infrastructure in place?

This is a much simpler problem than people make it out to be. How many people have heard of Picnic Health? Oh, one person. Not one of our— a couple of people. Not one of our companies, but what they do. And it's very cost effective. I have most of my medical data at Stanford. I've seen some UCSF people when I had my skiing accident. It's at Intermountain in Utah. They'll collect all this data and put it into a form that's actually usable, not just PDFs. I sort of can know what my BP and my blood glucose was 10 years ago and see it as a graph. So there's simple solutions. Now, they don't solve the problem of— and I don't know how many people heard— the Chinese army goes behind the big Equifax hack when the data for so many— credit card data for so many people got exposed. I think fundamentally most systems are hackable, and so we have to worry. When you put data together like Picnic Health does— and there's a couple of companies doing that kind of thing, and good efforts there are needed. But there's a much, much better solution. How many people have heard of a company called Nebula Genomics? Two hands, three hands. What they do is put the data, and they mostly deal with genomic data. They put your genome in the blockchain. They can't access it, but the consumer can. Consumer can then, through the system, permission them to sell it to Pfizer or whoever wants their data with the consumer's permission. I think that is ultimately the only real solution to data security. Put it in some form of blockchain. It doesn't have to be the Bitcoin blockchain. Make it only permissible by the end user. It's also a great point of integration because data has to come together from every medical or non-medical wearable, Apple Watch, HealthKit source you have data in and put it someplace where the user has control over it. And hacking, let me just say, is extremely difficult. Let's say they had $300 million, uh, 300 million people's worth of data in a record like this. If I'm a hacker, I could hack it and get a lot of data. What's it worth? Billion dollars? Billions of dollars? If I can hack the blockchain, I can steal $100 billion like that. So which one would I rather steal? The bigger prize, all of Bitcoin or your health data? I directly take cash. So you have this insurance, like if somebody hacks the system, there's much better things to steal than your personal data. But it is so far proven unhackable. So relatively, with relative certainty, it's very, very hard to hack today. And we know that quantum key distribution, other technologies are coming along to make it even more secure. So there is a solution to that, just no health system wants to give their data to the patient and for the patient to be the integration point for it and put it in a place outside the reach of the health system. In a secure blockchain. It's one of the few places the blockchain is extremely valuable because you can distribute— it's distributed trust as opposed to trusting some party, whether it's your bank or your healthcare provider or somebody else.

Vinod, we're out of time.

Great.

I talk a lot.

So I really want to thank you because this is the fourth time you've been here and this is really a lot of fun for me.

Well, it's always fun to talk about to an audience like this of I'm a techie nerd, so I love talking to technical people mostly. Hopefully no financial people here. But there's a lot of room for more data, more sensors, and what we can do and the insights we can derive from data. Thank you all very much.