AI's Human Factor | Stanford's Dr. Fei-Fei Li and OpenAI's Mira Murati on AI Safety

Hi everyone. Welcome to Grey Matter, the podcast from Greylock where we share stories from company builders and business leaders. I'm Heather Mack, head of editorial at Greylock. Today we're discussing how safety and ethical considerations must be factored into the development and deployment of artificial intelligence technology. Greylock general partner Reid Hoffman interviews Stanford professor Dr. Fei Fei Li, who is the co director of the center for Human Centered AI, and Mira Muradi who is the Chief technology officer of OpenAI. This interview took place during Greylock's Intelligent Future event, a day long summit featuring some of the leading experts and entrepreneurs working in artificial intelligence. You can watch the video of this interview on our YouTube channel and you can read a transcript on our website greylock.com now here's Reid Hoffman with Dr. Fei Fei Li and Mira Moradi.

While we'll be covering a bunch of things on safety, which is a highly relevant thing, especially in this kind of new foundational models universe because both Fei Fei and Mira are accomplished technologists, massively beyond the scope of safety, both building amazing things and have been part of the historic contributions. I thought we'd start with a kind of a more broad question which is what's currently most exciting you? So Fei Fei in AI, right? Fei Fei, I'll start with you, Mir, I'll go to you and then we will dive into safety.

Well, so many things are exciting me, but I guess I'll just say something that's on my mind. Right now we are finishing the review process of a paper that is kind of coming out which is redefining a North Star for robotics. I feel like this moment working in robotic learning feels a bit like back in the days of imagenet, days where we're really imagining what can be done that would make robotics dream come true. And this particular paper that we are hopefully, fingers crossed we're in the rebuttal review stage, will lay out a benchmark of 1,000 robotic tasks that are inspired by actual human activities. So it's a scale we've never seen in robotic research anywhere.

Is it too much of a presage to say what like one or two of the most interesting of those tasks are.

Yep.

We'Re going to turn off the.

Okay, sure, yes, one or two. Actually all of the thousand tasks comes from American Labor Bureau survey of American time usage and their equivalent of European government agency where we look at what humans do in their daily activity. And we actually did quite a bit of study on what people want for robots to help. For example, most people do not want robot to open Your Christmas gift. Apparently, we still want to do that ourselves, but almost everybody want the robot to clean the toilet, so that's ranked really high. Packing kids lunch is actually ranked fairly high.

Interesting.

We're talking about real human activities instead of, you know, how in robotics we tend to see toy examples?

Yes, exactly.

This is going to be 1000 real tasks.

Awesome. Yeah.

What's most exciting to me these days is seeing how far we can push the paradigm that's driving all AI development in the field, which is, you know, this combination of large neural networks with a ton of data and a vast amount of compute. And in the past few years, we've seen this formula drive a ton of progress in AI research. We saw that with GPT3 and then again with Codex and the first generation of Dall? E. So from OpenAI's perspective, we're trying to build these general systems that can think of the world in a similar way that humans do. So systems that have kind of a robust concept of the world. So if an AI system sees an avocado or reads the word avocado, the concept that gets triggered is exactly the same. And it's exciting to see that we've got systems that have achieved some sort of linguistic competency and understanding of visual concepts, and we'll continue to push this paradigm ahead.

So let's shift to safety, because both of your organizations, in different ways, are kind of some of the most important industry leaders on what we encounter with thinking about AI safety. And so, Mira, why don't we start with you, just so we mix it up a little bit. Not always go in this order, but will vary, say, a little bit about OpenAI's approach, what kind of being, this kind of novel organization organized around a 501C3 and organizations like this, how what you're trying to do to identify what safety is and how to create those norms, both in your own action, but also catalyzing other industry.

As you know, our mission is to build a general system and figure out how to deploy that beneficially to the world. And, you know, they're just one word, but beneficially. But figuring out how to robustly do that is actually an immense challenge. Challenge. It's hard to predict the future. It is hard to predict all the ways in which these systems might create harmful biases or other risks that we can't even imagine. But at the very least, we can try to get as much understanding as possible, gather as much knowledge as possible, and leave the options open. And that's OpenAI strategy. We are trying to to deploy these systems continuously, but in a controlled way. That means an API. So GPT3 was first deployed through an API to a small group of users, and then eventually we broaden up access as we understood how to get a good handle on the risks. But I think it's really hard. And that's why, that's actually part of the reason why we decided to deploy GPT. Because if you are in the research lab, you may think this is going to be the most prominent risk based on what we see. But when the model comes into contact with the real world, that gets tested and we found out in several times that we've deployed that we were wrong. And the most prominent risk was something else. For example, with GPT3, we were convinced that that misinformation was going to be the most important risk. And it is very important. But in practice, we saw that spam was actually a much bigger risk that we had to focus on. And the same thing happened with Dall E as well. So I think it's really important for these models to come in contact with real users, with the real world and understand where the friction is, where the limitations are, and iteratively building mitigations. And the mitigations we're building are probably not future proof, but it's a place to start and it gives us enough knowledge of where to go. But at the same time, we need to think about how the complexity increases as the models become more and more capable. So, for example, with language models, now we oversee the output of the model by a human for sensitive use cases. And that's not something that would scale with more powerful and advanced models. So then we have to come up with techniques to help humans evaluate the output of these models. And OpenAI has been working with other language model developers to coordinate on some of these standard practices to figure out how to deploy language models safely.

Fei, Fei, same question, but as opposed to OpenAI, human centered artificial Intelligence Institute, role of universities with industry, governments, plus the work you guys are doing.

Yeah, great question, Reid. Safety is one of those words like health. Everybody wants it, but it's really hard to define it. So wearing the hat of the Co Director of staff, Stanford Human Centered AI Institute, which we call Stanford High, we think a lot about what we really want for AI, for future AI, and we come to the word human centeredness. We focus on infusing the human centeredness into every stage of AI research, development, education, policy, work. Obviously we're university, so we don't deploy products, but we hope that what we Would inspire as applications will have a impact in productization downstream. So AI is not one thing. AI, you know, designing AI system is, is really stages of work decisions. And we believe that every stage of this AI development we need to infuse the ethics and human centered values into this simplest way to put it. How do we define a problem? For example, is your goal to use, you know, replace humans without consideration of all the social implications or augment human capability? That is, before you write a single line of code, you already are thinking about human values. The data, where does it come from? How do you collect it, how do you ensure data integrity, how do you annotate it? There is a whole bunch of, you know, from fairness to privacy to just a whole bunch of issues and considerations. Then the algorithm itself, you know, is it safe, is it secure, does it, you know, is it biased? And then the decision making using the algorithm, the inference, the human, how does it assist the humans or you know, inform the humans? So every stage of AI development needs human consideration. And Stanford High is really trying to embody that process. And in this, it's not even a side product, A central product of that process is the people we educate, the students, the undergrads, the graduate students. When they come out of this years of working or learning from Stanford High's courses and lab work, they become technologists or business leaders or policy thinkers who understand the human centeredness of AI.

So let's now dive a little deeper in each specific organization. So for Dall E and GPT3, the front end of the API is in part to understand what the possible risk flows look like so that you can begin to train those. How do you take that information and iterate to a beyond human kind of safety model? How do you take that and also help kind of lead the way in kind of thinking about this safety within the industry.

So for GPT3, for example, initially we opened up access to use cases that we felt we had the right mitigations in place. So that means very, very, almost like on the first few few days the allowed use cases were around search as well as classification. But we were not quite comfortable with open ended generation. And so we worked with industry experts from different domains as well as other researchers to red team the model a bit further. We worked with other trusted users as well to understand possible ways in which the model would fail, the expectations of the user, and there are many. So from there we try to build in mitigations, both from the model perspective, but also the tools that come after deployment from the model Perspective. One of the Central Things with GPT3, but also other generative language models is the fact that the model will make up stuff and it will not admit when it doesn't have expertise in a specific topic or when it doesn't know the answer. So obviously that's a problem in other ways. It could also mislead you with the answer. And so we wanted to figure out how do we make the model more robust, more reliable? And we used the feedback that we got from our users on the API and we used reinforcement learning with human feedback to make the instruct GPT models, which is a series of models that are far more reliable and do the thing that the operator actually wants them to do. So not only they're more helpful and safe, but they're actually useful. And so there are default models in the API today and our users prefer them rather than the base model. So this is one way where we use deployment to actually make the models safer, more reliable and more effective as well. And it is very interesting because it's the first time that safety moves from the theoretical domain in, into the practical domain and it merges with capabilities. And by doing that, it kind of forces a standard in the industry because not only it's safe, but it's actually more effective and more useful.

And then similarly, what are the things where Stanford High has pioneered some things, and specifically, I'm gesturing at the ethics review board, but go anywhere you like with the question.

Yeah, just to follow up, right. It's actually been a very encouraging trend to see that companies are really thinking about the ethics and the safe ways of developing and releasing products. In the meantime, in academia we are also thinking, for example, a typical way of putting some guardrails in our research activity is traditionally, if the research involves human subjects, we have IRB review boards. This has been critical across academia, in US universities and also international universities that a lot of medical research and human subject related research gets reviewed by a board of expert faculty and so on. But when AI is starting to become a really big chunk of university research, we don't have an equivalent of irb. At the beginning it was okay, it's theoretical, it's just. But very quickly we realized, wait a minute, there's so many examples of AI research, for example, face recognition. We see globally the biases and the harms of bias. So how do we, how do we put guardrails in the design of AI research and to ensure even the most technical researchers and students have in mind the ethics and the social impacts of their work? So at hai, one of our functions is to facilitate multidisciplinary research. And Reid, you have been a board member of our institute, so we talk a lot about that. It's really important we encourage researchers to do interdisciplinary research, but we need a new way, a new IRB to guide our researchers to think about ethical and social implications. So HAI was, as far as we know, the first organization, university organization in the country that formed what we call an ethics and Society Review board, ESR that reviews all grant applications. And as we assign grants to our researchers, we ensure that the grants work with the ESR board to articulate the understanding and potentially mitigation solutions of the social and ethical issues of the research. So that's one example of how much we put that emphasis in. Another example is our work with the policymakers. We work directly with regulators, policy makers and frankly, just civil society, you know, different people, multi stakeholders, to hear and to communicate the implications of AI, to hear their thoughts, to facilitate dialogues. We particularly focus on bringing industry and civil rights organizations, federal and state and local governments and policymakers to the same table. Create a forum, a safe, neutral forum to have these conversations because we all care about innovation and guardrails and we need this dialogue.

Add a little bit about the national research cloud too, because it's part of enabling universities and other entities like this in order to be able to fully participate.

Yeah. So one of the things that we recognize is the magic, one of the magic source of America in the past century is our ability as a country, as a people to innovate. We have an incredible ecosystem thanks to our entrepreneurial world, our higher education, the labs, the researchers. Also the role federal government has played in incentivizing research, whether it's NASA or DARPA or NSF and all that. But as AI took off in the past decade, what we observe is that the resources are quickly concentrated in a few companies with compute and data and talent, the three critical resources for doing incredible AI innovation, which is great. We see incredible things coming out of OpenAI, Microsoft, coming out of Google, DeepMind coming out of Facebook. But that's not enough to lift the entire nation to educate more AI talents to stand up in the face of global competition. We need to ensure this ecosystem remains healthy so that it's not just concentrated in industry per se. And right now, under Biden administration, the White House and the Congress have mandated a task force study group, a 12 person task force to figure out what this national research cloud will look like. I'm actually one of the members of this Task force, along with other people from industry, academia and government. And hopefully very soon we'll come up with our rapport and push this bill forward to establish this national resource.

Samira, I'm going to ask you. I have obviously 1000 questions for both of you. And depending on what the questions are from the audience, I may ask some more. But I'm going to ask you the last question before I turn to the audience, which is one of the things that GPT3 and Dall? E have both done is shown a path for really amplifying human creativity. It isn't just the downsides that is part of the question around safety and so forth, but also the, in a sense, helping us be more human, be more creative. Say a little bit about the lessons from Dall? E and GPT3 and what their potential is for this human amplification.

Yeah, Exactly. So from GPT3, early on, we were surprised to see the ability of the model to generate creative and even touching poetry. So one of the prompts that we gave GPT3 was to generate a poem in the style of Pablo Neruda that talked about Maxwell's equations. And we were quite surprised that it had this ability to pick up on the elements of Maxwell's foundational electromagnetic equations, but also do that in the style of the love poems of Pablo Neruda. So that was really beautiful. And a lot of people were playing with poetry and the creative side of GPT3. And I think we saw that even more with Dall? E. Dall? E, maybe because it's images and also the form factor in which we made it available through Dall? E labs, everyone was just having so much fun with it. Even at OpenAI, we would spend spend hours just generating Dall? E images. And this really just shows how technologies like Dali can democratize high quality creation of images and ideas and can push them so much further. And often we get asked, well, does this in some way dilute the human creation, the original human creation, and what happens in the future? There is this almost instinctive human reaction to protect our own original creations. And I think that if we look back in history, it's actually not so different from what happened in 16th and 17th century, where there weren't that many people that could afford paintings. And so things were quite binary. You were either Rembrandt or nothing. And there wasn't so much of a nuanced appreciation. It was either a great painting or not. And so I think as we get tools like DALL E or GPT3, maybe there's going to be a more nuanced appreciation for this co creation and a different appreciation for the original human creations. For example, today you can have a skilled artist that went to art school that can try to create a replica of the Night Watch. And to my undiscerning eye it will look great. But we still value real Rembrandt differently. But you can say, okay, this is sort of elitist point of view. What about the broader impact? And I think that's actually not so different from the effect of globalization because it's really an exchange of ideas, a cultural exchange. And of course there are sort of unwanted and undesired effects of globalization, but overall it does create more diversity and it does create more prosperity. And we've seen this in history with Western Europe, the 19th century being one of the most fertile and diverse from liberalization of cultural exchange. And by contrast, after the collapse of Roman Empire, it was quite the opposite. And so there is actually an interesting book about this called Creative Distraction that talks about this perceived fear that human talent will be diluted in a way, but actually if you look at it long term and the global effect. The global effect is one of diversification, one where we end up with more ideas in total more perspective and we will continue to develop more information and create artistically, scientifically and also in a social context.

And I think we have time for one question from the audience. I did also ask that last one because if you haven't had a chance to play with dall e and do your own printout, it's one of the things that OpenAI has helped provision us with. So it's a fun experiment. I think we have a question here. If I don't, there's another one.

Hello. Mira, just to the last point that you alluded to, you know, first, it's very heartening, the thoughtfulness and thoroughness of, you know, preempting and mitigating potential impact of launching some of these technologies. And I'm wondering, some of the effects cannot be simulated until you put that out there in the real world and probably at scale. So how do you think about, you.

Know, the.

Risks and issues at large scale, societal effects and potential second order effects as well? And what do you think is the role that we as technology companies can play in that? And probably related to that is Fei. Fei, you mentioned governing boards as an oversight. And I'm wondering how do you address the tension between the process interaction and the pace of innovation? We were colleagues in Cloud AI, if you remember, we've had oversight Boards internally and things get slightly slowed down as a result of. So would love to hear your thoughts, how you think about this.

Sorry. Actually, Mira, why don't you start?

You're right. So with deployment, we're just scratching the surface and we're putting these models out there, seeing what people are doing with them, seeing the risks and building tools to mitigate them. But more importantly, bringing the feedback back to the model development stage to build models that are more robust. And I think the alignment technique is really core to that. But more broadly, it's very important that we collectively are thinking about the governance systems of AI because obviously the developers of these models need to be a diverse group of people. But even that is not enough. And we have to think about AI, sort of like electricity. And so we have to think about the deployment and the governance of these systems more broadly in a global setting and bring in input from all different fields.

Yeah, and also great question about innovation versus the regulation or guardrails. I think we all get that question and it's always wonderful to think about it. First of all, I don't believe they're mutually exclusive. They're at odds with each other. I do believe as everything else in life, we have to strike the right balance. If we go to the extreme route that there should be zero guardrails, innovation might come out to fail us all. So it's not necessarily a world we want to live in. And of course, the other side of the extreme is also not good. In the case of AI, I think it's time for every organization, from individual researchers all the way to the government to have that dialogue to think about how this can be done and if it's done right, actually good guardrails can encourage good innovation. I'll give you one example of smart camera machine learning work in health care environment, which is part of my lab's work where we actually talking about safety. We actually use cameras to help doctors to monitor safety of patients. Now the intention is all great, but immediately we get into privacy issues, we get into bossware issues. Who are you watching? What are you watching? This comes back to us as an important social human feedback to push us to innovate more. My team of students who are used to just doing deep learning algorithm are now using a differential privacy algorithm to push for privacy protected machine learning. And then suddenly they realize differential privacy algorithms are too slow. It's never used in videos. We're using cameras. Suddenly that guardrail push, the regulatory push, is actually incentivized innovation. We're publishing papers that's pushing differential privacy machine ML algorithms to deal with the large scale data. So this is a perfect example of when we care about these guardrails. They come back to motivate us to innovate better technology. And I think there's plenty of examples in AI that we can do that and everybody is better off if we do that.

And with that, you can see why both Mira and Fit are people that I also constantly learn from. And so let's give them a hand.

Thank you.

Thank you.

Thank you guys.

That concludes this episode of Gray Matter. If you like what you hear and want to hear more like it, please subscribe wherever you get your podcasts and we encourage you to rate Gray Matter on your favorite streaming platform. We sincerely appreciate your feedback. You can also find all content on our website, greylock.com, and you can follow us on Twitter reylockvc. I'm Heather Mack and thanks for listening.