Anca Dragan: Human-Robot Interaction and Reward Engineering | Lex Fridman Podcast #81

The following is a conversation with Anka Dragon, a professor at Berkeley working on human robot interaction algorithms that look beyond the robot's function in isolation and generate robot behavior that accounts for interaction and coordination with human beings. She also consults at Waymo, the autonomous vehicle company, but in this conversation she is 100% wearing her Berkeley hat. She is one of the most brilliant and fun roboticists in the world to talk with. I had a tough and crazy day leading up to this conversation, so I was a bit tired, even more so than usual. But almost immediately as she walked in, her energy, passion and excitement for human robot interaction was contagious. So I had a lot of fun and really enjoyed this conversation. This is the Artificial Intelligence podcast. If you enjoy it, subscribe on YouTube, review it with five stars on Apple Podcast, support it on Patreon, or simply connect with me on Twitter lexfriedman spelled F R I D M A N. As usual, I'll do one or two minutes of ads now and never any ads in the middle that can break the flow of the conversation. I hope that works for you and doesn't hurt the listening experience. This show is presented by Cash App, the number one finance app in the App Store. When you get it, use code lexpodcast Cash App lets you send money to friends, buy Bitcoin, and invest in the stock market with as little as $1. Since Cash App does fractional share trading, let me mention that the order execution algorithm that works behind the scenes to create the abstraction of fractional orders is an algorithmic marvel. So big props to the Cash App engineers for solving a hard problem that in the end provides an easy interface that takes a step up to the next layer of abstraction over the stock market, making trading more accessible for new investors and diversification much easier. So again, if you get Cash App from the App Store or Google Play and use the code lexpodcast, you get $10. And Cash App will also donate $10 to the first, an organization that is helping to advance robotics and STEM education for young people around the world. And now here's my conversation with Anka Dragan. When did you first fall in love with robotics?

I think it was a very gradual process and it was somewhat accidental actually, because I first started getting into programming when I was a kid and then into math and then into I decided computer science was the thing I was going to do. And then in college I got into AI and then I applied to the Robotics Institute at Carnegie Mellon and I was coming from this little school in Germany that nobody had heard of but I had spent an exchange semester at Carnegie Mellon. So I had letters from Carnegie Mellon. So. So that was the only. MIT said no, Berkeley said no, Stanford said no. That was the only place I got into. So I went there to the robotics institute and I thought that robotics is a really cool way to actually apply the stuff that I knew and love like optimization. So that's how I got into robotics. I have a better story how I got into cars, which is I used to do mostly manipulation in my PhD, but now I do kind of a bit of everything application wise, including cars. And I got into cars because I was here in Berkeley while I was a PhD student still for RSS 2014. Peter Beal organized it and he arranged for it was Google at the time to give us rides in self driving cars. And I was in a robot and it was just making decision after decision the right call and it was so amazing. So it was a whole different experience. Right. Just. I mean manipulation is so hard, you can't do anything. And there it was.

Was that the most magical robot you've ever met? So like for me to meeting Google self driving car for the first time was like a transformative moment. Like I had two moments like that, that and Spot Mini. I don't know if you met Spot Mini from Boston Dynamics. I felt like, I felt like I fell in love or something like it. Because I thought I know how a Spot Mini works. Right. It's just. I mean there's nothing truly special. It's. It's great engineering work. But the anthropomorphism that went on into my brain that came to life like it had a little arm and it like. And it looked at me. He, she looked at me. You know, I don't know. There's a magical connection there. And it made me realize wow, robots can be so much more than things that manipulate objects. They can be things that have a human connection. Do you have. Was the self driving car the moment like was there a robot that truly sort of inspired you?

That was. I remember that experience very viscerally riding in that car and being just wowed. I. I had the. They gave us a sticker that said I rode in a self driving car and it had this cute little firefly on.

Yes.

And. Or a logo or something.

Oh that was like the smaller one. Like the.

Yeah, the really cute one. Yeah. And. And I put it on my laptop and I had that for years until I finally changed my laptop out.

And you know what about if we walk back you mentioned optimization like what beautiful ideas inspired you in math, computer science, early on? Like, why get into this field? Seems like a cold and boring field of math. Like, what was exciting to you about it?

The thing is, I liked math from very early on. From fifth grade is when I got into the Math Olympiad and all of that.

Oh, you competed too?

Yeah. This Romania is like our national sport, understand? So I got into that fairly early. And it was a little maybe too, just theory with no kind of. I didn't kind of had a. Didn't really have a goal. And other than understanding, which was cool. I always liked learning and understanding, but there was no, okay, what am I applying this understanding to? And so I think that's how I got into more heavily into computer science because it was kind of math meets something you can do tangibly in the world.

Do you remember like the first program you've written?

Okay, the first program I've written, I kind of do. It was in Cube Basic in fourth grade.

Wow.

And it was drawing like a circle.

Graphic square.

Yeah, that was. I don't know how to do that anymore. But in fourth grade that's the first thing that they taught me. It was like you could take a special. I wouldn't say it was an extracurricular. It was in a sense an extracurricular. So you could sign up for, you know, dance or music or programming. And I did the programming thing and my mom was like, what?

Why did you compete in program like these days? Romania probably. That's like a big thing. There's a programming competition. Was that, did that touch you at all?

I did a little bit of the computer science Olympian, but not, not as seriously as I did the math Olympian. So it was programming. Yeah, it's basically here's a hard math problem, solve it with a computer. It's kind of the deal.

Yeah, it's more like Algorithm.

Exactly, it's always algorithmic.

So again, you kind of mentioned the Google self driving car, but outside of that. Oh, what's like who or what is your favorite robot real or fictional that like captivated your imagination throughout? I mean, I guess you kind of alluded to the Google Slides, right? The Firefly was a magical moment. But is there something else?

There wasn't the Firefly in there. It was, I think that was the Lexus, by the way. This was back then, but yeah. So good question. Okay. My favorite fictional robot is Wall E. And I love how amazingly expressive it is. I'm personally thinks a little bit about expressive motion. Kinds of things. You're saying with you can do this. And it's a head and it's the manipulator. And what does it all mean? I like to think about that stuff. I love Pixar. I love animation.

I love Wall E. Has two big eyes, I think, or no.

Yeah, it has these cameras and they move. So. Yeah, that's. It's, you know, it goes woo. And then it's super cute. Yeah, it's. You know, the way it moves is just so expressive. The timing of that motion, what it's doing with its arms and what it's doing with these lenses is amazing. And so I've really liked that from the start. And then on top of that, sometimes I share this. It's a personal story I share with people or when I teach about AI or whatnot. My husband proposed to me by building a Wall E and he actuated it so it. Seven degrees of freedom, including the lens thing. And it kind of came in and it had the. He made it have like a, you know, the belly box opening thing. So it just did that and then it spewed out this box made out of Legos that open slowly and then bam. Yeah, yeah. It was quite, quite. It set a bar.

That could be like the most impressive thing I've ever heard. Okay.

That's a special connection to Wall E. Long story short, I like Wall E because I like animation and I like robots and I like, you know, the fact that this was. We still have this robot to this day.

What. How hard is that problem, do you think, of the. The expressivity of robots, like the. With the Boston Dynamics? I never talked to those folks about this particular element. I've talked to them a lot, but it seems to be like almost an accidental side effect for them that they weren't. I don't know if they're faking it. They weren't trying to. Okay. They do say that the. The gripper on it was not intended to be a face. I don't know if that's an honest statement, but I think they're legitimate.

It probably is.

So do we automatically just anthropomorphize anything we can see about a robot? So the question is, how hard is it to create a Wall E type robot that connects so deeply with us humans? What do you think?

Really hard. It depends on what settings. So if you want to do it in the. This very particular narrow setting where it does only one thing and it's expressive, then you can get an animator, you can have Pixar on call, come in, design some trajectories There was. Anki had a robot called Cozmo where they put in some of these animations. That part is easy, right? The hard part is doing it not via these kind of handcrafted behaviors, but doing it generally autonomously. I want robots. I don't work. Just to clarify, I used to work a lot on this. I don't work on that quite as much these days. But the notion of having robots that when they pick something up and put it in a place, they can do that with various forms of style. Or you can say, well, this robot is succeeding at this task and is confident versus it's hesitant versus maybe it's happy or it's disappointed about something, some failure that it had. I think that when robots move, they can communicate so much about internal states or perceived internal states that they have. And I think that's really useful and an element that we'll want in the future. Because I was reading this article about how kids are. Kids are being rude to Alexa because they can be rude to it. And it doesn't really get angry, right? It doesn't reply in any way. It just says the same thing. So I think there's at least for that, for the correct development of children, it's important that these things. You kind of react differently. I also think you walk in your home and you have a personal robot, and if you're really pissed, presumably robots should kind of behave slightly differently than when you're super happy and excited. But it's really hard because I don't know the way I would think about it and the way I thought about it when it came to expressing goals or intentions for robots, it's. Well, what's really happening is that instead of doing robotics where you have your state and you have your action space, and you have your space, the reward function that you're trying to optimize. Now you kind of have to expand the notion of state to include this human internal state. What is the person actually perceiving? What do they think about the robots? Something or rather. And then you have to optimize in that system. And so that means you have to understand how your motion, your actions, end up sort of influencing the observer's kind of perception of you. And it's very hard to write math about that, right?

So when you start to think about incorporating the human into the state model, apologize for the philosophical question, but how complicated are human beings? Do you think? Like, can they be reduced to. To a kind of almost like an object that moves and maybe has some basic intents or Is there something. Do we have to model things like mood and general aggressiveness and I mean, all these kinds of human qualities or like game theoretical qualities, like what's your sense?

How complicated is.

How hard is the problem of human robot interaction?

Yeah. Should we talk about what the problem of human robot interaction is?

Yeah, this is what is human robot interaction.

And then talk about how that. Yeah, so. And by the way, I'm going to talk about this very particular view of human robot interaction. Right. Which is not so much on the social side or on the side of the. How do you have a good conversation with the robot? What should the robot's appearance be? It turns out that if you make robots taller versus shorter, this has an effect on how people act with them. So I'm not talking about that, but I'm talking about this very kind of narrow thing, which is if you want to take a task that a robot can do in isolation, in a lab, out there in the world, but in isolation, and now you're asking what does it mean for the robot to. To be able to do this task for presumably what its actual end goal is, which is to help some person. That ends up changing the problem in two ways. The first way it changes the problem is that the robot is no longer the single agent acting. You have humans who also take actions in that same space. Cars navigating around people, robots around an office navigating around the people in that office. If I send the robot over there in the cafeteria to get me a coffee, then there's probably other people reaching for stuff in the same space. And so now you have your robot and you're in charge of the actions that the robot is taking. Then you have these people who are also making decisions and taking actions in that same space. And even if the robot knows what it should do, and all of that just coexisting with these people, getting the action. Actions to gel well, to mesh well together. That's sort of the kind of problem number one. And then there's problem number two, which goes back to this notion of if I'm a programmer, I can specify some objective for the robot to go off and optimize and specify the task. But if I put the robot in your home, presumably you might have your own opinions about, well, okay, I want my house clean, but how do I want it cleaned, and how should robot move, how close to me it should. Should come, and all of that. And so I think those are the two differences that you have. You're acting around people and what you should Be optimizing for should satisfy the preferences of that end user, not of your programmer who programmed you.

Yeah. And the preferences thing is tricky. So figuring out those preferences, be able to interactively adjust to understand what the human is. So really it boils down to understand the humans in order to interact with them, in order to please them.

Right.

So why is this hard?

Yeah, why is understanding humans hard? So I think there's two tasks about understanding humans that in my mind are very, very similar. But not everyone agrees. So there's the task of being able to just anticipate what people will do. We all know that cars need to do this, right? We all know that. Well, if I navigate around some people, the robot has to get some notion of, okay, where is this person going to be? So that's kind of the prediction side. And then there's what you are saying, satisfying the preferences, right? So adapting to the person's preferences, knowing what to optimize for, which is more this inference side, this what does this person want? What is their intent, what are their preferences? And to me, those kind of go together because I think that at the very least, if you can understand, if you can look at human behavior and understand what it is that they want, then that's the key enabler to being able to anticipate what they'll do in the future. Because I think that we're not arbitrary. We make these decisions that we make, we act in the way we do because we're trying to achieve certain things. I think that's the relationship between them. Now, how complicated do these models need to be in order to be able to understand what people want? So we've gotten a long way in robotics with something called inverse reinforcement learning, which is the notion of someone acts, demonstrates how they want the thing done.

What is inverse reinforcement learning?

You briefly said it right? So it's the problem of take human behavior and infer reward function from this. So figure out what it is that that behavior is optimal with respect to. And it's a great way to think about learning human preferences in the sense of you have a car and the person can drive it and then you can say, well, okay, I can actually learn what the person is optimizing for. I can learn their driving style. Or you can have people demonstrate how they want the house clean and then you can say, okay, this is, this is, I'm getting the trade offs that they're making. I'm getting the preferences that they want out of this. And so we've been successful in robotics somewhat with this, and it's based on a very simple model of human behavior. It was remarkably simple, which is that human behavior is optimal with respect to whatever it is that people want. Right. So you make that assumption, and now you can kind of inverse through. That's why it's called inverse. Well, really optimal control, but. But also inverse reinforcement learning. So this is based on utility maximization in economics. Back in the 40s, von Neumann, Morgenstein were like, okay, people are making choices by maximizing utility. Go. And then in the late 50s, we had loose and shepherd come in and say, people are a little bit noisy and approximate in that process. So they might choose something kind of stochastically with probability proportional to how much utility something has. There's a bit of noise in there. This has translated into robotics and something that we call Boltzmann rationality. So it's an evolution of inverse reinforcement learning that accounts for human noise. And we've had some success with that too. For these tasks where it turns out people act noisily enough that you can't just do vanilla. The vanilla version, you can account for noise and still infer what they seem to want based on this. Then now we're hitting tasks where that's not enough.

What are examples of task.

So imagine you're trying to control some robot that's. That's fairly complicated. You're trying to control a robot arm because maybe you're a patient with a motor impairment and you have this wheelchair mounted arm and you're trying to control it around or. One test that we've looked at with Sergey and our students did is a lunar lander. So I don't know if you know this Atari game. It's called Lunar Lander. It's really hard. People really suck at landing the thing. Mostly they just crash it left and right. Okay, so this is the kind of task where imagine you're trying to provide some assistance to a person operating such a robot, where you want the kind of the autonomy to kick in, figure out what it is that you're trying to do, and help you do it. It's really hard to do that for, say, Lunar Lander, because people are all over the place. And so they seem much more noisy than really irrational. That's an example of a task where these models are kind of failing us. And it's not surprising because. So we talked about the 40s utility, late 50s, sort of noisy. Then the 70s came and behavioral economics started being a thing where people were like, no, no, no, no, no. People are not rational people are messy and emotional and irrational and have all sorts of heuristics that might be domain specific. And they're just. Just a mess.

It's a mess.

So what does my robot do to understand what you want? And that's why it's complicated. For the most part, we get away with pretty simple models until we don't. And then the question is, what do you do then? And I had days when I wanted to pack my bags and go home and switch jobs, because it feels really daunting to make sense of human behavior enough that you can reliably understand what people want, especially as robot capabilities will continue to get developed. You'll get these systems that are more and more capable of all sorts of things, and then you really want to make sure that you're telling them the right thing to do. What is that thing? Well, rooted in human behavior.

So if I just sat here quietly and tried to understand something about you by listening to you talk, it would be harder than if I got to say something and ask you and interact and control. Can the robot help its understanding of the human by influencing the behavior, by actually acting?

Yeah, absolutely. So one of the things that's been exciting to me lately is this notion that when you try to. When you try to think of the robotics problem as, okay, I have a robot and it needs to optimize for whatever it is that a person wants it to optimize, as opposed to maybe what a programmer said, that problem we think of as a human robot collaboration problem in which both agents get to act, in which the robot knows less than the human, because the human actually has access to, at least implicitly, to what it is that they want. They can't write it down, but they can talk about it. They can give all sorts of signals they can demonstrate. But the robot doesn't need to sit there and passively observe human behavior and try to make sense of it. The robot can act too. And so there's these information gathering actions that the robot can take to sort of solicit responses that are actually informative. So, for instance, this is not for the purpose of assisting people, but with kind of back to coordinating with people in cars and all of that. One thing that DORSA did was so we were looking at cars being able to navigate around people and you might not know exactly the driving style of a particular individual that's next to you, but you want to change lanes in.

Front of them, navigating around other humans inside cars.

Yeah, good clarification. Question. So you have an autonomous car and it's trying to navigate the road around human driven vehicles. Similar things, ideas apply to pedestrians as well. But let's just take human driven vehicles. So now you're trying to change a lane. Well, you could be trying to infer the driving style of this person next to you. You'd like to know if they're in particular if they're sort of aggressive or defensive, if they're going to let you kind of go in or if they're going to not. And, and it's very difficult to just, you know, if you think that if you want to hedge your bets and say, ah, maybe they're actually pretty aggressive, I shouldn't try this. You kind of end up driving next to them and driving next to them. Right. And then you, you don't know because you're not actually getting the observations that you get. The way someone drives when they're next to you and they just need to go straight is kind of the same regardless of their aggressive or defensive. And so you need to enable the robot to reason about how it might actually be able to gather information by changing the actions that it's taking. And then the robot comes up with these cool things where it kind of nudges towards you and then sees if you're going to slow down or not. And if you slow down, it sort of updates its model of you and says, okay, you're more on the defensive side. So now I can actually.

That's a fascinating dance. And so that's so cool that you could use your own actions to gather information. That's. That feels like a totally open, exciting new world of robotics. I mean, how many people are even thinking about that kind of thing?

A handful of us, I guess.

It's rare because it's actually leveraging human. I mean, most roboticists I've talked to, a lot of, you know, colleagues and so on, are kind of being honest, kind of afraid of humans because they're messy and complicated.

Right, I understand. Going back to what we were talking about earlier, right now we're kind of in this dilemma of, okay, there are tasks that we can just assume people are approximately rational for and we can figure out what they want, we can figure out their goals, we can figure out their driving styles, whatever. Cool. There are these tasks that we can't. So what do we do? Do we pack our bags and go home? And this one, I've had a little bit of hope recently and I'm kind of doubting myself because what do I know that 50 years of behavioral economics hasn't figured out, but maybe it's not really in contradiction with the way that field is headed. But basically, one thing that we've been thinking about is instead of giving up and saying people are too crazy and irrational for us to make sense of them, maybe we can give them a bit the benefit of the doubt. And maybe we can think of them as actually being relatively rational, but just under different assumptions about the world, about how the world works, about they don't have. When we think about rationality, implicit assumption is, oh, they're rational and they're all the same assumptions and constraints as the robot, right? This is the state of the world. That's what they know. This is the transition function. That's what they know. This is the horizon. That's what they know. But maybe, maybe the kind of this difference, the way the reason they can seem a little messy and hectic, especially to robots, is that perhaps they just make different assumptions or have different beliefs.

So, I mean, that's another fascinating idea that this. Our kind of anecdotal desire to say that humans are irrational, perhaps grounded in behavioral economics, is, is that we just don't understand the constraints and the rewards under which they operate. And so our goal shouldn't be to throw our hands up and say they're irrational, is to say, let's try to understand what are the constraints, what it.

Is that they must be assuming that makes this behavior make sense. Good life lesson, right? Good life lesson.

That's true. It's just outside of robotics, that's just good to. That's communicating with humans. That's just a good assume that you just don't. Empathy, right?

Maybe there's something you're missing, and it especially happens to robots because they're kind of dumb and they don't know things and oftentimes people are sort of super irrational and that they actually know a lot of things that robots don't sometimes, like with the lunar lander, the robot knows much more. So it turns out that if you try to say, look, maybe people are operating this thing, but assuming a much more simplified physics model because they don't get the complexity of this kind of craft or the robot arm with seven degrees of freedom with these inertias and whatever, maybe they have this intuitive physics model, which is not. This notion of intuitive physics is something that is studied actually in cognitive science folks like Josh Dannenbaum, Tom Griffith's work on this stuff. What we found is that you can actually try to figure out what. What physics model kind of best explains human actions. Then you can use that to correct what it is that they're commanding the craft to do. They might be sending the craft somewhere, but instead of executing that action, you can take a step back and say, according to their intuitive. If the world worked according to their intuitive physics model, where do they think that the craft is going? Where are they trying to send it to? And then you can use the real physics, the inverse of that, to actually figure out what you should do so that you do that instead of where they were actually sending you in the real world. And I kid you not, it worked. People land the damn thing in between the two flags and all that. So it's not conclusive in any way, but I'd say it's evidence that maybe we're kind of underestimating humans in some ways when we're giving up and saying they're just crazy noisy.

So then you try to explicitly try to model the kind of worldview that they have.

That they have. That's right. That's right. There's not too. I mean, there's things in behavioral economics too, that, for instance, have touched upon the planning horizon. So there's this idea that there's bounded rationality, essentially, and the idea that, well, maybe we work under computational constraints. And I think kind of our view recently has been take the Bellman update in AI and just break it in all sorts of ways by saying, state. No, no, no. The person doesn't get to see the real state. Maybe they're estimating somehow transition function. No, no, no, no, no. Even the actual reward evaluation, Maybe they're still learning about what it is that they want. When you watch Netflix and you have all the things and then you have to pick something, imagine that the AI system interpreted that choice as, this is the thing you prefer to see. How are you going to know? You're still trying to figure out what you like, what you don't like, et cetera. So I think it's important to also account for that. So it's not irrationality, doing the right thing under the things that they know.

Yeah, that's brilliant. You mentioned recommender systems. What kind of. And we were talking about human robot interaction. What kind of problem spaces are you thinking about? So is it robots, like wheeled robots with autonomous vehicles? Is it object manipulation? Like when you think about human robot interaction in your mind, and maybe I'm sure you can speak for the entire community of human robot interaction, but like, what are the problems of interest here?

Is.

And does it. You know, I kind of think of open domain Dialogue as human robot interaction. And that happens not in the physical space, but it could just happen in. In the virtual space. So where's the boundaries of this field for you when you're thinking about the things we've been talking about?

Yeah, so I try to find kind of underlying. I don't know what to even call them. I try to work on, I might call what I do the kind of working on the foundations of algorithmic human robot interaction and trying to make contributions there. And it's important to me that whatever we do is actually somewhat domain agnostic when it comes to, is it about autonomous cars or is it about quadrotors, or is it about the same underlying principles apply? Of course, when you're trying to get a particular domain to work, you usually have to do some extra work to adapt that to that particular domain. But these things that we were talking about around. Well, how do you model humans? It turns out that a lot of systems need to core benefit from a better understanding of how human behavior relates to what people want and need to predict human behavior. Physical robots of all sorts and beyond that, I used to do manipulation. I used to be picking up stuff, and then I was picking up stuff with people around. And now it's sort of very broad when it comes to the application level, but in a sense very focused on, okay, how does the problem need to change? How do the algorithms need to change when we're not doing a robot by itself, you know, emptying the dishwasher. But we're stepping outside of that.

I thought that popped into my head just now on the game theoretic side of things. You said this really interesting idea of using actions to gain more information. But if we think of sort of game theory, the humans that are interacting with you, with you. The robot. I'm taking the identity of the robot.

Yeah, I do that all the time.

Yeah. Is they also have a world model of you and you can manipulate that. If we look at autonomous vehicles, people have a certain viewpoint. You said with the kids, people see Alexa as a. In a certain way. Is there some value in trying to also optimize how people see you as a robot? Is that. Or is that a little too far away from the specifics of what we can solve right now?

So we can both. Right. So it's really interesting. And we've seen a little bit of progress on this problem, on pieces of this problem. So you can, again, it kind of comes down to how complicated does the human model need to be. But in one piece of work that we were looking at, we just said, okay, there's these parameters that are internal to the robot and what the robot is about to do, or maybe what objective, what driving style the robot has, or something like that. And what we're going to do is we're going to set up a system where part of the state is the person's belief over those parameters. And now when the robot acts, the person gets new evidence about this robot internal state. And so they're updating their mental model of the robot, right? So if they see a card that sort of cuts someone off, they're like, oh, that's an aggressive card. They know more, right? If they see a robot head towards a particular door, they're like, oh, the robot's trying to get to that door. This thing that we have to do with humans to try to understand their goals and intentions, humans are inevitably going to do that to robots. Then that raises this interesting question that you asked, which is, can we do something about that? This is going to happen inevitably, but we can be more confusing or less confusing to people. It turns out you can optimize for being more informative and less confusing if you have an understanding of how your actions are being interpreted by the human, how they're using these actions to update their belief. And honestly, all we did is just Bayes rule, basically, okay, the person has a belief, they see an action, they make some assumptions about how the robot generates its actions, presumably as being rational. Because robots are rational, it's reasonable to assume that about them. And then they incorporate that new piece of evidence in the Bayesian sense, in their belief, and they obtain a posterior. And now the robot is trying to figure out what actions to take such that it steers the person's belief to put as much probability mass as possible on the correct parameters.

So that's kind of a mathematical formalization of that. But my worry, and I don't know if you want to go there with me, but I talk about this quite a bit. The, the kids talking to Alexa disrespectfully worries me. I worry in general about human nature. Like I said, I grew up in Soviet Union, World War II. I'm a Jew too. So with the Holocaust and everything, I just worry about how we humans sometimes treat the other. The. The group that we call the Other, whatever it is through human history, the group that's the other has been changed faces. But it seems like the robot will be the other, the other, the. The next, the other. And one thing is, it feels to me that robots don't get no respect.

They get shoved around.

Shoved around. And is there one at the shallow level for a better experience? It seems that robots need to talk back a little bit, like my intuition says. I mean, most companies from sort of Roomba autonomous vehicle companies might not be so happy with the idea that a robot has a little bit of an attitude. But I feel, it feels to me that that's necessary to create a compelling experience. Like we humans don't seem to respect anything that doesn't give us some attitude that. Or like a mix of mystery and attitude and anger and that threatens us subtly, maybe passive aggressively. I don't know. It seems like we humans need that do. What are your. Is there something. You have thoughts on this?

Okay, I'll give you two thoughts on this.

Okay, sure.

One is we respond to someone being assertive, but we also respond to someone being vulnerable. So I think robots, My first thought is that robots get shoved around and bullied a lot because they're sort of tempting and they're sort of showing off or they appear to be showing off. And so I think going back to these things we were talking about in the beginning of making robots a little more. A little more expressive, a little bit more like that wasn't cool to do. And now I'm bummed. Right. I think that that can actually help because people can't help but anthropomorphize and respond to that. Even that though the emotion being communicated is not in any way a real thing. And people know that it's not a real thing because they know it's just a machine. We still interpret, you know, we watch. There's this famous psychology experiment with little triangles and kind of dots on a screen and a triangle is chasing the square and you get really angry at the darn triangle because why is it not leaving the square alone? So that's. Yeah, we can't help. So that was the first thought.

The vulnerability. That's really interesting. I think of like being. Pushing back, being assertive as the only mechanism of getting, of forming a connection, of gaining respect. But perhaps vulnerability, perhaps there's other mechanisms that are less threatening.

Yeah.

Is there how you.

Well, I think a little bit, yes. But then this other thing that we can think about is it goes back to what you were saying that interaction is really game theoretic. So the moment you're taking actions in a space, humans are taking actions in that same space. But you have your own objective, which is you're a car. You need to get your passenger to the destination and then the human nearby has their own objective which somewhat overlaps with you, but not entirely. You're not interested in getting into an accident with each other, but you have different destinations and you want to get home faster and they want to get home faster. And that's a general sum game at that point. And so that's, I think that's what treating it as such is kind of a way we can step outside of this kind of mode that, where you try to anticipate what people do and you don't realize you have any influence over it while still protecting yourself because you're understanding that people also understand that they can influence you. And it's just kind of back and forth is this negotiation which is really, really talking about different equilibria of a game. The very basic way to solve coordination is to just make predictions about what people will do and then stay out of their way. And that's hard for the reasons we talked about, which is you have to understand people's intentions implicitly, explicitly, who knows? But somehow you have to get enough of an understanding of that to be able to anticipate what happens next. And so that's challenging. But then it's further challenged by the fact that people change what they do based on what you do. Because they don't plan in isolation either, right? So when you see cars trying to merge on a highway and not succeeding, one of the reasons this can be is because they look at traffic that keeps coming. They predict what these people are planning on doing, which is to just keep going, and then they stay out of the way because there's no feasible plan. Any plan would actually intersect with one of these other people. So that's bad. So you get stuck there. So now kind of, if you start thinking about it as. No, no, no, actually these people change what they do depending on what the car does. Like if the car actually tries to kind of inch itself forward, they might actually slow down and let the car in. And now taking advantage of that, well, that, you know, that's kind of the next level. We call this like this underactuated system idea where under actresses in robotics. But it's kind of, it's. You don't. You influence these other degrees of freedom, but you don't get to decide what they do somewhere.

Seen you mention it, this, the human element in this picture as under actuated. So you know, you understand under actuated robotics is, you know, that you can't fully control the system.

You can't go in arbitrary directions in.

The configuration space under your Control?

Yeah, it's a very simple way of under actuation where basically there's literally these degrees of freedom that you can control and these degrees of freedom that you can't, but you influence them. And I think that's the important part is that they don't do whatever, regardless of what you do, that what you do influence is what they end up doing.

I just also like the poetry of calling human robot interaction an underactuated robotics problem. And you also mentioned sort of nudging. It seems that there. I don't know, I think about this a lot. In the case of pedestrians, I've collected hundreds of hours of videos. I like to just watch pedestrians. And it seems that it's a funny hobby. Yeah, it's weird because I learn a lot. I learn a lot about myself, about our human behavior from watching pedestrians. Watching people in their environment, basically crossing the street is like you're putting your life on the line, you know, I don't know, tens of millions of time in America every day is. People are just like playing this weird game of chicken when they cross the street. Especially when there's some ambiguity about the right of way that has to do either with the rules of the road or with the general personality of the intersection based on the time of day and so on. And this nudging idea, I. I don't. You know, it seems that people don't even nudge. They just aggressively take. Make a decision. Somebody. There's a runner that gave me this advice. I sometimes run in. In the street and you know, not in the street, on the sidewalk. And he said that if you don't make eye contact with people when you're running, they will all move out of your way.

It's called civil inattention.

Civil inattention. That's a thing. Oh, wow, I need to look this up. But it works. What is that? My sense was if you communicate like confidence in your actions that you're unlikely to deviate from the action that you're following, that's a really powerful signal to others that they need to plan around your actions. As opposed to nudging, where you're sort of hesitantly, the hesitation might communicate that you're now you're still in the dance, in the game that they can influence with their own actions. I've recently had a conversation with Jim Keller, who's a sort of this legendary chip architect, but he also led the autopilot team for a while. And his intuition that driving is fundamentally still like a ballistics problem. Like you can ignore the Human element that it's just not hitting things. And you can kind of learn the right dynamics required to do the merger and all those kinds of things. And then my senses, and I don't know if I can provide sort of definitive proof of this, but my sense is like an order of magnitude or more, more difficult when humans are involved. Like it's not simply object collision avoidance problem. What's. Where does your intuition, of course, nobody knows the right answer here, but where does your intuition fall on the difficulty, fundamental difficulty of the driving problem when humans are involved?

Yeah, good question. I have many opinions on this. Imagine downtown San Francisco. Yeah, it's crazy busy. Everything okay now take all the humans out, no pedestrians, no human driven vehicles, no cyclists, no people on little skills, electric scooter zipping around, nothing. I think we're done. I think driving at that point is done. We're done. There's nothing really that's still needs to be solved about that.

Well, let's pause there. I, I think I agree with you like and I think a lot of people that will hear will agree with that, but we need to sort of internalize that idea. So what's the problem there? Because we might not quite yet be done with that because a lot of people kind of focus on the perception problem. A lot of people kind of map autonomous driving into. How close are we to solving, being able to detect all the, you know, the, the drivable area, the objects in the scene. Do you see that as. How hard is that problem? So your intuition there behind your statement was we might have not solved it yet, but we're close to solving basically the perception problem.

I think the perception problem, I mean, and by the way, a bunch of years ago this would not have been true. And a lot of issues in the space were coming from the fact that, oh, we don't really, you know, we don't know what's, what's where. But I think it's fairly safe to say that at this point, although you could always improve on things and all of that, you can drive through downtown San Francisco if there are no people around. There's no really perception issues standing in your way there. I think perception is hard, but yeah, we've made a lot of progress on the perception side to undermine the difficulty of the problem. I think everything about robotics is really difficult. Of course, I think that, you know, the planning problem, the control problem, all very difficult. But I think what makes it really. Yeah, it might be, I mean, you know, and I picked downtown San Francisco adapting to, well now it's snowing. Now it's no longer snowing. Now it's slippery in this way. Now the dynamics part I could imagine being still somewhat challenging.

But no, the thing that I think worries us and our intuition is not good. There is the perception problem at the edge cases, sort of downtown San Francisco. The nice thing it's not actually, it may not be a good example because.

Because you know what to. What you're getting from. Well, there's like crazy construction zones and all.

Yeah. But the thing is you're traveling at slow speeds so like it doesn't feel dangerous to me. What feels dangerous is highway speeds when everything is to us humans super clear.

Yeah. I'm assuming LIDAR here, by the way. I think it's kind of irresponsible to not use lidar. That's just my personal opinion. I mean depending on your use case. But I think like, you know, if you, if you have the opportunity to use LIDAR in a lot of cases you might not.

Good. Your intuition makes more sense now. So you don't think vision.

I really just don't know enough to say, well, vision alone. What, you know, what's like there's a lot of. How many cameras do they have? Is that. How are you using them? I don't know. There's all, there's all sorts of details. I imagine there's stuff that's really hard to actually see. You know, how do you deal with glare? Exactly what you were saying. Stuff that people would see that you don't. I think I have more of my intuition comes from systems that can actually use LIDAR as well.

Yeah. And until we know for sure it makes sense to be using LiDAR. That's kind of the safety focus, but then the sort of the. I also sympathize with the Elon Musk statement of lidar as a crutch. It's a fun notion to think that the things that work today is a crutch for the invention of the things that will work tomorrow. Right. It's kind of true in the sense that we want to stick to the comfort zone. You see this in academic and research settings all the time. The things that work force you to not explore outside, think outside the box. That happens all the time. The problem is in the safety critical systems you kind of want to stick with the things that work. So it's, it's a, it's an interesting and difficult trade off in the, in the, in the case of real world sort of safety critical robotic systems. But so your intuition is just to clarify.

Yes.

How. I mean, how hard is this human element for, like, how hard is driving when this human element is involved? Involved? Are we years, decades away from solving it, but perhaps actually the year isn't the thing I'm asking. It doesn't matter what the timeline is, but do you think we're. How many breakthroughs are we away from in solving the human robot interaction problem to get this right?

I think it. In a sense, it really depends on. I think that we were talking about how. Well, look, it's really hard because anticipating what people do is hard. And on top of that, playing the game is hard, but I think we sort of have some of the fundamental understanding for that. And then you already see that these systems are being deployed in the real world, even, even driverless. Like there's, I think now a few companies that don't have a driver in the car in some small areas.

I got a chance to. I went to Phoenix and I. I shot a video with Waymo. I need to get that video out. People have been giving me slack. But it's incredible engineering work being done there and it's one of those other seminal moments for me in my life to be able to. It sounds silly, but to be able to drive without a. With a ride. Sorry, without a driver in the seat, I mean, that was an incredible. Robotics. I was driven by a robot and without being able to take over, without being able to take the steering wheel, that's a magical. That's a magical moment. So.