The Near Future of AI [Entire Talk] - Andrew Ng (AI Fund)

Welcome everybody to the Entrepreneurial Thought Leaders Seminar, a Stanford seminar for aspiring entrepreneurs. ETL is presented by stvp, the Engineering center, the Engineering Entrepreneurship center here at Stanford and basis, the Business association of Stanford Entrepreneurial Students. I'm Ravi Balani, a lecturer in the Management Science and Engineering Department at Stanford and the Director of Alchemist and Accelerator for Enterprise Startups. Today we are thrilled to welcome Andrew Ng to etl. How many people know Andrew? Okay, so Andrew really doesn't need an introduction, but we will give one anyways for those who don't. Andrew is truly a child of the world. He was born in the UK to parents who emigrated to the UK from Hong Kong, but was raised in Hong Kong and Singapore, went to Carnegie Mellon and very early on signaled that he was no ordinary student. He got three bachelor's degrees at Carnegie Mellon in Computer science, statistics and Economics and graduated at the top of his class, then went on to MIT where he got a Master's in Electrical Engineering and Computer science, then came over to the Left coast and got a PhD in Berkeley in Computer Science with a focus on artificial intelligence and reinforcement learning. Andrew is generally viewed as one of the preeminent thought leaders on AI today. He's the co founder and head of Google Brain and the former chief scientist at Baidu, where he built the company's AI group into thousands of people, several thousand people. But he's as passionate about AI as he is also about the development of you of students around the world. And I know there's a lot of love for Andrew. He's a former Associates professor and director of the Stanford AI Lab and currently an adjunct professor in Computer science at Stanford. How many people have taken one of Andrew's classes or want to take one of Andrew's classes? And he's a beloved professor here at Stanford, but he's also viewed as a beloved teacher to millions outside of Stanford. He's the co founder and chairman of Coursera, the world's largest MOOC platform. And through his online education work and his online AI education work, he's reached over 7 million people. He was listed as one of the world's hundred most influential people by time magazine in 2013. And today, Andrew is the Managing Director and Partner at the AI Fund, which is a startup studio building new AI companies from the ground up and is also the founder of Deep Learning AI. He focuses his time primarily on his entrepreneurial ventures, looking for the best ways to accelerate responsible AI practices in the larger global economy. There is fantastic content already online that Andrew has given, including A longer version of today's talk that you can find on YouTube. And so instead of reduplicating that, Andrew's going to give a teaser talk, a 10 minute discussion followed by we'll do a quick fireside chat, and then we're going to open it up for really interactive Q and A with you. So start thinking about your questions now because the time is going to fly by. But without further ado, please welcome Andrew.

Thanks a lot, Ravi. Thanks. Good to see everyone here. Can everyone in the back hear me? Okay, cool. Awesome. You know, I've taught CS229 in my machine learning class in this room many years, but all these years I taught in this room, I've never seen my face that big. First of all, what I'd like to do today is chat with you about opportunities in AI. So one of the difficult things to understand about AI is a general purpose technology similar to electricity, meaning it's not useful just for one thing, it's useful for a lot of different applications. If I were to ask you, what is electricity good for? It's almost hard to answer that because it's useful for so many different things. And AI is like that too. So one of the major trends that we've seen in the last year, few years is that prompting is revolutionizing AI application development. And I want to just dive a little bit deeper into this because I know this is an engineering class. I know many of you may be from an engineering background. I'm going to just go a little bit deeper than this, into this than I might otherwise. But if you were to say, want to build an AI system for many years, the typical approach is use supervised learning. So let's say I want to build a system to rate restaurant reviews as positive or negative sentiment. You know, then you would collect data, maybe that takes me a month. I would train an AI model, maybe that takes me a few months, find a cloud service to deploy my AI model, maybe that takes, you know, a few months. And so for the past, most of the past decade, a realistic timeline to build and Deploy a valuable AI system was, you know, maybe six to 12 months. But with prompting, the timeline is now very different. You can specify a prompt in minutes or hours and then deploy a system to production in just hours or days. And I know that probably many, or maybe most, maybe all of you will have played with large language models as a consumer too, like ChatGPT and Bard and Bing Chat, I think that in terms of startup opportunities, I'm excited about the use of large language Models, not as a consumer too, which is fantastic and exciting, I think. I use ChatGPT and Bard regularly, but instead the application of large language models as a developer tool because this is allowing a lot more AI applications to be built and dramatically lowering the barrier to building many applications. So I know that in this talk you don't normally have speakers, write code, but this is an engineering class, so let me actually show you exactly what I mean by that, right? It turns out that if I want to build an AI system today, this is all the code I need. And this means that, you know, if you take CS106 or something, learn to code right in a CS class with just a little bit of code Import AI OpenAI tools, load my key. I don't know. DST V lectures are great, you know, also made many friends, never written that before. And so hopefully this. Okay, thank goodness, got that right. Um, and so this is positive sentiment. And just in seconds, you know, that's all the code it takes now to build an AI system in code, to look at a piece of text and process it, to look at piece of email and route it, or to build a start, to build the beginnings of a chatbot. So over the last, I don't know, half year, one of my teams DeepLearning AI has been working with many of the AI tool builders to create short courses on how to use tools like I just showed you, because there are many AI applications that used to take me six months to build that I think any of you will now be able to build in one or two days. And this opens up the set of things that you could do and the set of prototypes you can build. And in fact, from a startup perspective, when it took us six months to build something, you know, what we do is have a product manager study it, do the user studies, make sure it's the right thing to build, then go build it. And after all that investment, it's like, boy, let's hope it works. But what I'm seeing with these very fast development times is if it takes you a couple days to build something, I'm seeing a lot more startups as well as big companies say, you know What, I have 10 ideas for features. I'm going to build all 10 things and then just ship them all and then we'll see how users use them or don't use them, and we'll just keep what sticks. Um, and this is a very different prototyping, much leaner methodology than I've seen startups use, you know, before prompting with One, one important caveat, which is that responsible AI is important. So don't do this, don't ship, you know, things that could cause harm. But we have a lot of applications like inspecting bits of metal in factories, you know, where there is really no harm, no risk of bias, where I think there's very fast shipping methodology lets us innovate very quickly in AI. So where are the opportunities? So the size of these circles shows what I think is the value of different AI technologies today. Um, supervised learning started to work really well about a decade ago at labeling things such as label this ad as is this something you're likely to click on or not? Or label this X ray with, you know, what's the medical diagnosis? And supervised learning for a single company like Google is worth more than $100 billion a year and there are millions of developers working on it. And it might even grow in the next three years to double, say so. Massive momentum, lots of applications to be figured out. And then generative AI is a new entrant where frankly the revenue, the value of the revenue from generative AI today is much smaller. But given the amount of interest, excitement and commercial interest, I think it will much more than double in the next few years. And three years is an artificially short time horizon. I think if we were to look out six years, if it continues to compound at this rate, maybe the value from generative AI will even start to approach that is supervised learning. But all that room for growth, the light shaded region for supervised learning or generative AI, which are probably the two most important tools today, are where there are a lot of opportunities for any of us to identify and build to concrete use cases. And what I hope you take away from this talk is AI technologies are general purpose technologies, meaning that they're useful for many different tasks. When supervised learning started to work well, about a decade ago, it actually took us a long time. It took us annoyingly long over the last decade, and it will take us annoyingly long over the next decade to figure out use cases for generative AI. But do you want to use this to make chips more efficient or for medical diagnosis or for education, product recommendations or something else? It takes. We're still figuring out concrete use cases for supervised learning. And even though we're not yet done doing that, we have another fantastic new tool, generative AI, that even further expands the set of things we can now do with AI. Um, and you know, one important caveat, which is there will be fads along the way. How many of you remember Lens, raise your Hand if you do. Wow. Almost no one. That's fascinating. So Lenza's revenues took off like that through last December. Um, it was this app that could let you upload a few pictures of yourself and draw a cool picture of you as an astronaut or a scientist or something. Um, and it was a really good, really hot product until last December, after which its revenues did that. And I think that's because Lenze was one of what will probably turn out to be multiple thin software layers built on top of someone else's very powerful API. That was a good idea. People liked it. But it wasn't a long term defensible business. And when I think about generative AI as a developer platform, I'm reminded of when, you know, Steve Jobs gave us the stone, right? Um, and shortly after, someone wrote an app that I paid 199 for to do this, to turn the phone into a flashlight. And this was also a good idea. It was a great product, but it just was not a defensible business either because it was a very thin software layer built on top of someone else's very powerful development platform. Um, but in the same way, after we got the iPhone, after we got the smartphone, someone else figured out how to build Uber, Airbnb and Tinder. Much longer term defensible, very valuable businesses that are still standing the test of time. And I think what those opportunities as well, to build long term valuable franchises businesses on top of generative AI. So where are the opportunities? So I felt years ago, but even more strongly now, that because of emerging AI technology, there are a lot of projects that are now possible that were not possible one or a handful of years ago. And I wound up starting AI Fund, which is a venture studio that sequentially works with entrepreneurs to start companies. We actually average about one startup a month now because I felt I previously, as Ravi mentioned, previously I had led AI teams in Google and Baidu, and having led AI teams in big tech, I couldn't see how you could possibly operate a team in a big tech company to pursue the very diverse, very different sets of opportunities that I saw and wanted to pursue. And starting different startups to pursue those valuable projects seem more efficient than having one company, even the big tech company, go after such a large set of resources. But having said that, I think AI and generative AI also offers a lot of opportunities for incumbent companies which often have a distribution advantage. Where exactly are the opportunities? So this is what I think of as the AI stack. At the lowest level is the hardware layer very valuable. But Also very capital intensive, needs a lot of resources to build and very concentrated. So I'm sure there'll be valuable startups built there, but I personally don't play there because of how capital intensive and how concentrated it is. There's a cloud infrastructure layer, also very capital intensive, very concentrated, very valuable. But at least when I build startups, I tend not to play there. The other layer that's interesting is the developer tooling layer. So what you just saw me do was use OpenAI as a developer tool and I see this space as hyper competitive. Look at all the startups chasing OpenAI. But there will be some mega winners. So whereas incumbents have a startup kind of a distribution advantage, I think for many startups having a technology advantage may give you a better shot at doing something meaningful there. So I personally tend to play it here only when we think we have a technology advantage because that buys us a better chance to be, to become one of the huge winners. And then with most ways of technology innovation, a lot of media attention, social media, what people tend to talk about is the tooling, the technology layer. There's one other layer that I think has got to be even more valuable and that's the application layer. Because in many ways the technology for the infra and tooling layer to be successful applications need to be built on top of them that generate even more revenue so that they can afford to play the infrastructure layer. And what I'm seeing is that there are a lot of opportunities at the application layer where the intensity of competition is not, frankly not nearly as high. Maybe just one example. I've been chatting a lot with the CEO of Me Know, which is a startup that applies AI to romantic relationship coaching. Right. And you know, I'm an AI guy. I feel like I don't know anything about romance. And if you don't believe me, you can ask my wife. She will confirm that I don't know anything about romance. But when we decided, when we had conviction that AI could be applied to relationships, we wound up partnering with Renate Nyborg, who's the former CEO of Tinder. And because she ran Tinder, she understands relationships in a very systematic way, more so than anyone else I know. And so with my team providing AI expertise and her providing relationship expertise, we're able to build you have pretty unique relationship mentoring application that we just announced a few weeks ago. And this may not probably know. Renate actually occasionally stops by Stanford campus and talks to Stanford students as part of her user product research. So it's possible to see her out. Just one last thing, I love to go to Q and A. Over the last few years, AI Fund, we've been tuning our process for building styles. I'm just going to share that with you. So we often start off with a lot of ideas, right? And one example of another startup we built was Bearing AI which uses AI for smart routing a very large ocean going vessel. So if you're a ship captain, should you sail at 20 knots or 22 knots? It's like, who knows? Most ship captains just make some decision, but because we're able to get global weather and ocean current data, we can make recommendations to ship captains for how to get there on time and use about 10% less fuel. But this idea was suggested to me by Mitsuyi, which is a major shareholder in a major shipping line that operates very large ocean going vessels. And this is one of those things. I would never have thought of this idea myself because, you know, like I've been on a boat, but what do I know about global maritime shipping? But Mitsuyi suggested this idea to me and we then validate the idea, make sure there's technical feasibility and the market need recruit a CEO. We are fortunate to find Dylan Kyle, who's a fantastic CEO with one successful exit before. And then we spent three months in our current process building a technical prototype with the CEO and doing deep customer validation. If it survives, 2/3 chance of surviving, 1/3 chance of not surviving, we then write a check in that allows the company to build, hire executives, build an MVP and off it goes to raise additional rounds of capital. And I think this is what we. And so Bearing AI. Well now it's actually there are now hundreds of ships on the high seas guided by bearing AI. Ships guided by bearing AI have sailed 75 million miles, which is the equivalent of going 3,000 times around the planet, and save about half a million dollars in fuel cost per ship per year. In addition to significant carbon emissions. I think we've saved about, I want to say about a million tons of CO2 emissions so far. But this kind of idea that I would never have come up with this idea myself, but I've learned that my swim lane is AI. But when I work with experts in other sectors, there are often these exciting opportunities that are very valuable. But frankly, how many groups in the world are experts in AI and shipping or experts in AI and relationships? I find that the competition intensity at the application layer is often much lower. And then just one last thing, kind of, you know, just full disclosure and something I hope all of you will do, too. My teams only work on projects that we think move humanity forward. Uh, responsible AI is important, and on multiple occasions, we've killed, and I will continue to kill projects that we may assess to be financially sound but based on ethical grounds. So, lots of exciting opportunities. Um, I think at Stanford, there are lots of great classes you can take in engineering and elsewhere to learn about that AI tech. And then when you find, you know, applications or. Or go play at the infra and tooling layer too. I think there are lots of opportunities, but I think there are. What I'm seeing is, frankly, my team at AI Fund with so many startup ideas. We use a task management software, we use Asana to track this huge list of ideas. And it's actually quite clear to me there are a lot more good ideas for AI businesses than, you know, people with the skill to work on them at this moment in time. So hopefully there'll be more than enough projects for everyone. All of you, all of us, to work on. Right, thank you.

I wanted to just start off with that closing statement that you made about how there's more opportunity than there are students with skills or people with skills to pursue them. And given that we have an audience full of students, I wanted to start off by mapping out advice for students that are entering into the university regarding AI. So if you want to pursue a career in AI right now, and let's say your child was entering Stanford, um, what advice would you give them in terms of how to spend their time?

Yeah, so, you know, there's one thing that's actually really worth doing when you're sent to students, which is take classes. Because it turns out that I feel like there's actually one pattern I see for both undergraduates and graduate students, including PhD students, which is there's so much exciting stuff to do, you just want to jump in and do it right. In fact, I've seen them undergrads in their freshman year, you know, try to join a research lab and start doing work in AI. That's okay. Nothing wrong with that. But it turns out that while project work is one way to learn, coursework is, I think, an even more efficient way to learn, especially when it comes to mastering the fundamentals, because professors will put a lot of work to organize the material in a way that's efficient to learn and digest. So I would say, you know, take classes in AI technology or in entrepreneurship and gain those skills. I've seen students jump in, and then if you are trying to work on a research lab without strong skills, you end up labeling data or something, which is fine, you learn some things, but you actually learn a lot from taking courses. And then in addition to that, after you start to master the foundational skills, after you know how to use AI technology, or then as you start to practice, find exciting use cases across campus. I do a lot of work over with people over in climate science or in healthcare to take my AI expertise and then marry it with a different discipline that I'm not expert in to find exciting applications. And hopefully that type of practice will help many of you find exciting projects to work on as well.

Do you need to take technical classes? Do you think you need to take computer science classes? If you want to pursue a career.

In AI, need is too strong, but I definitely encourage you to take technical classes. I think we're moving toward a world where frankly, at some future point I think everyone should learn to code. Or rather I think it'll be useful for everyone to learn how to code for a couple of reasons. Everyone has access to data. This is different than the world used to be even a few years ago. And especially with generative AI, your ability to get something to work is much higher than ever before. The barrier to entry is much lower than ever before. And so if you learn just a little bit of coding, the amount that you will accomplish is significantly greater than if you don't know how to code at all.

And are there any skills that separate out the great AI founders? I know AI right now it's a sea that's rising all boats. But if you separate out the great ones from the good ones, are there any salient skills that you notice that the great AI CEOs or founders have that the good ones don't?

Maybe. Since you said AI, I would say is often technical depth, it helps a lot. But I want to give a different answer if you say great founders around great AI founders. But I feel like AI is evolving rapidly and we definitely have lots of entrepreneurs that pitch the VCs without really knowing what they're doing. And the smart VCs can stiffen up quite quickly and it makes a huge difference. I think the technology unfortunately, you know, is like somewhat complicated for a lot of applications. So a team that actually knows what they're doing will execute an AI project 10 times faster, you know, than a team that doesn't. And 10 times is not a made up number. I literally see people take a year to do something like, oh boy, I know that other team would have done performed this level in two weeks or maybe a month. So for many AI startups, application startups, infrast startups, you kind of have to know what you're doing so it doesn't have to be you. If you're a technical co founder, maybe that's okay. And then second thing I see among many of the great founders is speed. I find that as a startup you'd be surprised when you hang up the great founders, the sheer speed of decision making. And you know, I sometimes talk to people from big companies and they'll say, oh, we move so fast. But when I kind of sit them side by side, how long does it take you to make this decision? I talk to a great founder. How long does it take my decision? Maybe. Here's one story. I was chatting with the meno CEO of Renata Nyborg, former CEO of Tinder. I was on the phone with her one day and she was making a major architecture decision. She was architecting this thing. You know, there are basically two architect, two major software architectures under consideration. And the team had laid it out, list out some pros and cons. So they got on the team with me and some of my friends and said, these are pros and cons. And then one of my CTO AI funded, I said, you know, we're not sure, but here's some reason we prefer architecture A. And then Renate said, okay guys, done, decision made, go and implement, you know, Architecture A. And after I thought, well, did Renate just make a massive engine decision in basically 30 seconds? And she did. And I realized afterward, I don't think there was a better way to make it because it's not as if, you know, if the company waited another week would have been a high quality decision. And if it was wrong, I'm sure they would fix it, you know, the next week. But until you've lived through the speed of a great business, most people, I know so many people that think their organizations are fast when you stack them up to the real speed of a fast moving CEO, they have never actually seen speed in their life. One important caveat do be responsible. I know that move fast and break things sometimes, you know, is the wrong approach. So tremendous speed when you are not being callous with, with people's lives and livelihood and things that could cause real harm. But so long as there's an important caveat of responsible AI, many of the great CEOs move faster than most people realize people can move.

And so let's just double click on that on this theme of responsible AI, just because I know this is a hot topic that maybe people are thinking about, which is you are clearly on the side of AI for good, for responsible AI. Many of your brethren, like Geoffrey Hinton and other famous leaders in the AI space, have come out and are concerned that the pace of AI development will become an existential threat to humanity. So much so that famously there was a petition signed by Elon Musk and Steve Wozniak and many thought leaders asking for the halting of the deepest foundational models of AI for us to sort, for society to sort of catch up. You did not sign that pledge. Can you share a little bit more detail about that? Was that a difficult decision for you to make? And can you share more details about why you didn't join them and what your philosophical view is regarding if AI poses an existential threat?

So I honestly don't see how AI poses any existential threat to the human race. We know AI can run them up. Self driving cars have crashed, leading to tragic loss of life, automated tradings, crash the stock market. So we know poorly designed software systems can have a dramatic impact and responsible AI is important. But recently I sought out people like Jeff and others that were concerned about the question of AI extinction. And I tried to understand why they thought this way. Some were worried about bad actor using AI to create a bioweapon. Others were worried about AI evolving in a way that inadvertently leads to human extinction, similar to how we as humans have led to the extinction of many species through simple lack of awareness sometimes that our actions could lead to that outcome. But when I tried to assess how realistic these arguments were, I found them to be vague and NonSpecific about how AI could kill us all. And I think that I found, frustratingly frankly, that trying to prove AI couldn't is akin to proving a negative. And I can't prove that superintelligent AI won't be dangerous. But I can't seem to find anyone that really knows exactly how it could be. And, but, but, but I do know that humanity has ample experience controlling many things far more powerful than any one of us, like corporations and nation states. And there are many things that we can't fully control that are nonetheless safe and valuable, like airplanes. You know, no one can control an airplane, it's buffeted around by winds and, and the pilot may make a mistake. But in the early days of aviation, airplanes killed many people. So we learned from those experiences, built safer aircraft, devised rules by which to operate them. And today most of us can step into airplane without fearing for our lives. And I think it will be like that too for AI. So I think the AI extinction I find to be very unfortunate what I'm seeing because you know, doing Some work in K12 education as well, what I'm seeing is that kind of really unfortunately I see high school students now considering working in AI and some will say AI seems exciting, but I heard it could lead to human extinction and I just don't want to be a part of that. And so I find that the overhyped AI extinction narrative is doing real harm. So I'm really concerned about that.

Thank you Andrew. One more question then I'm going to open it up which is I loved the detail on the low hanging fruit opportunities. I know that's on everybody's, all the entrepreneurs minds of what to pursue. And so I appreciated the attention and the presentation on that. I wanted to ask about what's going to be the next big technology shift in AI because things are changing so rapidly, especially as the models now are getting smaller and open sourced. It feels like we've already conquered language. Visual AI is getting very, very good. What's next? What are you seeing that's around the corner that others might not be aware of?

Yeah, you know what, done it about several months ago I was predicting visual AI is coming next. But now everyone's all right. Visual AI, guess you gotta come with something new. But in all seriousness, I think visual AI would be much more about the analysis of images rather than just generation of images. But I think we're at like the GPT2 moment for visual AI is not yet working but I think it'll work much better. And this will impact self driving cars for example, when we can finally solve problems in the Long tail. And then I think actually one other thing that I wrote about just today in a newsletter called the Batch is I think one thing that many people find controversial but I think is coming is the rise of edge AI. And I know this is controversial. Many of us were trained to write SaaS software, you know, lends itself nice to subscription business model. How do you even find people like how do you hire engineers to write desktop applications? Like who even does that anymore? But I think that because of, for various forces including privacy, I think that in the next few years we'll see more AI applications running at the edge, meaning on your laptop, on, on your cell phone. So I think that would be coming and then I think there'll just be a lot of work coming. The application layer as well.

Okay, I want to open it up to the students. You're the Reason why Andrew's here. You mentioned that on your slides you.

Put the potential from reinforcement learning, or the general value is a dot relative.

To the potential for unsupervised learning. Do you think there is still potential for generalist agents like GATO and other reinforcement learning models in society and in your AI stack for startups?

So, okay, so technically, last time Hmongers are trained using reinforcement learning and unsupervised learning and supervised learning. But leaving that aside, I feel like I'm not convinced that reinforcement learning is near a breakthrough moment, at least in the next small number of years. A lot of excitement about what we could do in reinforcement learning. Applied to robotics. A lot about CS faculty. Right. Chelsea Finn, Emma Brunskill, many others are doing exciting research there. But we do have a data problem. Um, so it turns out that text on the Internet sounds a lot like text on your documents. So we can learn from lots of text on the Internet to do really well on your text. Documents and images on the Internet look a little bit like images that you care about. So we have a lot of data. But because every robot is different, um, I'm struggling, many, many people are struggling to see how to get enough data to have the usual recipe of scaling up data and compute, reference, enforcement, learning. And people are working on it. Over the weekend at the CS faculty retreat, you know, there was a talk, I think, who gave this talk, Shoot. I'm thinking on how to do this, on early ideas of how to do this, but I think we're still a few years away from breakthroughs in those, breakthroughs in reinforcement learning. But it's a great research topic, by the way. Just because, you know, just because it's not working right now doesn't mean you shouldn't do research on it. So I think it's a great research topic.

So I just want to know your thoughts about what are the security concerns which is coming up by abusing that, Like LLM models, Like all these new attacks like prompt injections, data leakage, jailbreaking. So what's your thought around that? Like, how can we like safeguard against those kind of attack? Because it's just starting up this new technology, so I'm assuming there's more things which will come up.

Yes. So I think that for the near future there'll be a little bit of a cat and mouse thing going on. So I think I'm seeing different companies approach this with different tools to watch out for prompt injections for data leakage. Actually, Deepline AI is actually working with a Partner on some things that hopefully will announce very soon on a portfolio of tools. By the way, those of you that have not yet, you know, done it, go, go and fool around with prompt injections. See if you can get an LLM, you know, to do something. Well, don't, don't, don't do something actually harmful. But it's, I actually find it kind of intellectually interesting whenever I use an LLM to prompt it to see how robust the safeguards actually are. And, but, and if actually look at the older language models. A year ago it was super easy to get the older models, you know, frankly to give you detailed directions to do things that it should not give anyone detailed directions to do. But the more modern language models are much harder. But it's still sometimes possible. Um, sorry. Oh, but what I'm seeing as well for a lot of corporations, a lot of corporations because of these worries will ship internal facing product first because presumably, you know, if, if it says the wrong thing to your own employee, more understanding, less likely risk scandal and test products internally for quite a long time or even built capabilities for safe internal use before turning out to external use. But I do see different companies. Yeah. Different tools for trying to adjust these.

Terrific. Next question. Thank you so much.

Hi there, My name is Chai Nat.

And I'm an international student from Hong Kong. I'm curious to ask because, you know.

I'm hoping that after I graduate I can hopefully go back home to work closer with family.

But at the same time I feel.

Like by going back I'm closing a.

Lot of doors behind me because for example, in Hong Kong, for example, you can't access ChatGPT without a US number, which makes access to some of these resources really difficult. So I'm curious to see what are.

Your thoughts about navigating this complex modern landscape? Yeah, I don't want to comment on, I don't know, complicated. That's actually one thing I'm seeing. I've been to quite a few places in Asia recently and what I'm seeing is that many countries are developing surprisingly good capabilities for building large language model applications. The concentration of talent for generative AI deep tech is very concentrated in the San Francisco Bay area, I think because there are basically two teams that did a lot of the early groundbreaking work. You know, Google Brain, my former team and OpenAI and subsequently people left and started a lot of companies here in California Bay area. So I think that concentration of talent is very high. And it's interesting even when I'm in, you know, Seattle Great city, love the city. On weekends, you know, I hang out with friends, but the conversations are not about generative AI. Whereas here you kind of like if you go to a coffee shop. Actually, one of my friends was visiting from Taiwan, so he was hanging out with us for a week. They went back and he said, yeah, I went to a coffee shop. And, you know, there was no one talking about AI. That's so weird. So at least at this moment in time, there's really heavy concentration. But I see less the deep tech layer, but the application layer. I see that skill set developing quite quickly globally as well. Oh, and I think the opportunities in a lot of places will be local opportunities. So the shipping company that we built, we built it with a Japanese company that happens to operate global lines of shipping. So I think a lot of the businesses will be, you know, playing locally where that country or that geography is strong. Those businesses will be more efficient to build in places other than Silicon Valley, because where do I go to find a large seaport here to do that type of work? Hi, Andrew, thank you so much for your time. My name is Komal. I wanted to ask you if you think we'll ever reach a threshold on human dependence for AI or if you think it'll just continue to grow exponentially. So I think we already really, really depend on tech, right? Imagine if, you know, if the Internet were to shut down, I think people would die. I don't think that's exaggerated. I mean, but seriously, think about how we get food, supply chain, healthcare. If the Internet were to shut down, I think that will lead directly to what will happen to our water system. Right. Healthcare system. And I think that technology is very useful. And so long as the supplies remain reliable, I feel like it's okay to depend on technology. I mean, heck, I wish, I don't know, without dependence on agriculture system, how many of us would be able to farm and hunt enough food to keep ourselves alive. Maybe, maybe we could do it, but it's pretty challenging. So I think dependence on tech seems to keep on growing for a while.

But do you think there'll be a moment where there's a difference in that relationship? Not just in degree, but in kind? The famous singularity point where we don't even know what. We don't even know about how technology is developing. Do you think that will occur?

Yeah, the technological singularity is one of those hypey things that I don't even know what it means. So it's exciting science fiction, but as an engineering scientist, I don't know how to talk about. Turns out there are a few terms in AI that are vague and undefined, but there are a lot of emotions, a lot of excitement about it. And I don't really know how to think about those things in a systematic, rational way. But I think I'll take actually there's actually one thing. I think that our relationship with technology is changing rapidly today. I Probably use chat, GPT4 or Bing or bot pretty much every day now. And so the workflow of many people have change. I think it keeps on changing.

And do you have a view I know this is also might be more of one of these sort of hot topics that's not substantive but on the consciousness of AI, that AI, will it become conscious?

Yeah. So the thing about consciousness is an important philosophical question, but I don't know of any test for whether something is conscious or not. So I think it's important philosophy and philosophy is important. But as an engineer and scientist, I don't know. There is no definition for what is conscious or not and thus we can kind of debate it at length. And there's actually one other formula for hype which is if someone comes up with a very simple definition for consciousness. So someone says, oh, if you can recognize yourself in the mirror, you're conscious. I made that up. It's not a good definition of consciousness, but you're aware of yourself, see yourself in a mirror. Then it's actually pretty easy to get a robot to recognize itself in a mirror. And then you can generate newspaper headlines saying AI has achieved consciousness. What it did for your kind of, you know, silly little. For your very small definition of consciousness. But that gets misinterpreted by the broader public for a grander statement than it is. So I see some of that hype in AI as well.

Thank you. Next question. Hi. Earlier you outlined the AI stack and recently we've seen a lot of cool things coming out of like Nvidia, intel and other like chip companies. I'm curious on what your thoughts are on what companies like AWS and Google, like in the infrastructure layer need to do in order to make like AI and enterprises and business really effective and possible.

Sure. Boy. So there's a lot going on in that space. By the way, you mentioned intel and Nvidia. I wouldn't. I think I'm actually seeing really exciting work from AMD as well. I've been pretty impressed by the Mi 200, Mi 250. I'm excited about Mi 300 GPUs coming out as well. And I think the ROCM stack is becoming, you know, not parity of Cuda, but better than most people give them credit for. But in terms of aws. So it turns out that if you were to use a lot of the LLM startup tools, the switching cost is actually pretty low. So if you were to, you know, start with one LLM API call, if you want to switch to a different LLM provider, the number of lines of code is actually pretty low. So there are low searching costs. But it turns out that Azure and Google Cloud and AWS are fantastic businesses because once you build on any of these clouds, you know, the switching costs tend to be very high because you have so many API host integrations. So that's why I think that a lot of the startups selling API calls still have, you know, some work to do to find a business model that may be somewhat more defensible. I think that OpenAI's ChatGPT enterprise, that feels like a more defensible business than just selling API calls. By the way, Sam was actually a Stanford undergrad. He actually interned. He's a curious in my lab. So a lot of Stanford routes. He's a smart guy. I'm sure he'll figure out confidently. He'll figure out some good directions. Yeah, and I think AWS and GCP and Azure are all racing to continue to develop LM capabilities and make it easier to use and bring in more customers. And yeah, it's very, very dynamic space.

And as AI gets democratized, it feels like things are shifting more towards computer and data as predictors of success. If that's the case, do you think the locus of innovation shifts from academia to industry where the companies are going to really be dominating at the forefront of AI?

Yeah. So what I'm seeing now is that there's a subset, but there's a small subset of things that are easier to do in a big tech company which are the ones that require massive compute resources. And I do think people's perceptions are distorted because frankly I've been on a big tech companies before.

Right.

So I understand marketing and big tech companies, but standard big tech company marketing is look, you need the data, you need to compute, only we have it. Why don't you just give up and don't compete with us? Right. And, or even, or come apply for a job and come work with us. That is, this has been the explicit PR strategy of at least one big tech company because I know you know what was discussed internally exactly at that big tech company. So I would say don't buy into that marketing message. It is true that there is a subset of work that requires massive capital training in very large foundation models that is much easier to do in a big tech company than an academia like Stanford. But that's a small subset of all the, all the happenings in AI. And there's plenty of work at Stanford at the application layer. It turns out because of scaling laws, we're actually pretty good at predicting what will happen for very large models by training on more model sized models. So very good scientific work can be done at much smaller models. And then also, you know, I routinely run kind of, you know, models on my laptop for inference. Like I don't know, when I'm on an airplane. You can run like the 7 billion llama model on your laptop. Right. And so there's actually a lot of stuff that you, you could run on on your own personal computer.

Thank you. Next question. Thank you, Andrew. So from AI expert and also the invest investor's perspective, so what AI driven.

Healthcare applications do you see have the.

Great potentials to have the breakthrough in the future and what challenges and obstacles.

Should be we be aware of?

Thank you.

Yeah, so boy, there's a lot of complexity to that question. So I feel like, I feel like a lot of healthcare people tend to focus on the diagnostics and the treatment. So I think lots of opportunities there. I think that the revenue model is to be sorted out. So we've seen, you know, Pear and IKEA struggle in the public markets, kind of bankruptcy kind of levels almost. So I think prescriptive digital therapeutics is definitely going through challenges. But what's the recipe for shipping AI products and you know, in the payer provide ecosystem, what will payers be willing to pay for? I think that many businesses are sorting that out. I think that will work. And there's actually one other huge set of opportunities in healthcare that I think tend to be underappreciated, which is operations instead of the medical stuff, things like scheduling, you know, who should scheduling the MRI machine or doing kind of patient management systems. I think those type of healthcare operations have fear of regulatory hurdles and I think is also a rich set of opportunities. And then lastly there's the go to market question of do you want to go to market in the US or in other countries where the regulatory hurdle could be very different depending on the US fortunately doesn't have as great a shortage of doctors as some other places. And those there are therefore other places that are more amenable to responsible but still easier adoption of AI than the United States. Okay, I have a super quick question.

You mentioned that your team at the.

AI Fund has so many ideas for.

AI applications that you have a whole asana of them. What exactly is your process for generating these ideas?

Oh, thanks, Aozak.

And you have 30 seconds. Sure.

We like working with subject matter experts that deeply understand the domain. It turns out that there are a lot of people in the world, including like CEOs of Fortune 500 companies, but really a lot of people that really understand the domain have thought deeply about something for months or even a couple of years. And when we get together with them, they're sometimes very happy to share their idea with us because they've been looking for someone to validate or falsify it and also to help them build it. So we actually got a lot of ideas, some internal, but a lot from subject matter experts that have just not yet had an AI build upon there.

Terrific. That's fantastic. Thank you, Andrew, so much for sharing your insights. Lots of love. Thank you for sharing your insights with Stanford's ETL course, MSE 472 and the students all around the world. Everybody, next week we're going to be joined by Stanford Professor Kathleen Eisenhart here at ETL physically in person. Professor Eisenhart is also the author of Simple Rules. You can find that event and other future events in this ETL series on the Stanford E Corner YouTube channel. And you'll find even more of our videos, podcasts and articles about entrepreneurship and innovation at Stanford E Corner. That's Ecorner Stanford Edu. Thank you, everybody. Thank you, Andrew.

Thanks, Bobby.