Russ Tedrake: Underactuated Robotics, Control, Dynamics and Touch | Lex Fridman Podcast #114

The following is a conversation with Russ Tedric, a roboticist and professor at MIT and Vice President of Robotics Research at Toyota Research Institute or tri. He works on control of robots in interesting, complicated, under actuated, stochastic, difficult to model situations. He's a great teacher and a great person. One of my favorites at mit. We'll get into a lot of topics in this conversation, from his time leading MIT's DARPA Robotics Challenge team to the awesome fact that he often runs close to a marathon a day to and from work barefoot. For a world class roboticist interested in elegant, efficient control of underactuated dynamical systems like the human body, this fact makes Russ one of the most fascinating people I know. Quick Summary of the ads three sponsors Magic Spoon Cereal, BetterHelp and ExpressVPN please consider supporting this podcast by going to MagicSpoon.com Lex and using code Lex at checkout. Going to BetterHelp.com Lex and signing up at ExpressVPN.com LexPod click the links in the description. Buy the stuff, get the discount. It really is the best way to support this podcast. If you enjoy this thing, subscribe on YouTube, review it with five stars on Apple Podcast, subscribe, support it on Patreon, or connect with me on Twitter lexfriedman. As usual, I'll do a few minutes of ads now and never any ads in the middle that can break the flow of the conversation. This episode is supported by Magic Spoon Low Carb Keto Friendly Cereal I've been on a mix of keto or carnivore diet for a very long time now that means eating very little carbs. I used to love cereal. Obviously most have crazy amounts of sugar, which is terrible for you. So I quit years ago. But Magic Spoon is a totally new thing. 0 sugar, 11 grams of protein and only 3 net grams of carbs. It tastes delicious. It has a bunch of flavors. They're all good.

But if you know what's good for.

You, you'll go with cocoa, my favorite flavor, and the flavor of champions. Click the magicspoon.com lex link in the description. Use code lex at checkout to get the discount and to let them know I sent you. So buy all of their cereal. It's delicious and good for you.

You won't regret it.

The show is also sponsored by BetterHelp, spelled H E L P. Help. Check it out at betterhelp.com lex they figure out what you need and match you with a licensed professional therapist in under 48 hours. It's not a crisis line, it's not self help. It is professional counseling done securely online. As you may know, I'm a bit from the David Goggins line of creatures and so have some demons to contend with, usually on long runs or all nighters full of self doubt. I think suffering is essential for creation, but you can suffer beautifully in a way that doesn't destroy you for most people. I think a good therapist can help in this, so it's at least worth a try.

Check out the reviews. They're all good.

It's easy, private, affordable, available worldwide. You can communicate by text anytime and schedule weekly audio and video sessions. Check it out@betterhelp.com Lex this show is also sponsored by ExpressVPN. Get it at expressvpn.com Lexpod to get a discount and to support this podcast. Have you ever watched the Office? If you have, you probably know it's based on a UK series also called the Office. Not to stir up trouble, but I personally think the British version is actually more brilliant than the American one, but both are amazing. Anyway, there are actually nine other countries with their own version of the Office. You can get access to them with no geo restriction. When you use ExpressVPN, it lets you control where you want sites to think you're located. You can choose from nearly 100 different countries, giving you access to content that isn't available in your region. So again, get it on any device@expressvpn.com lexpod to get an extra 3 months free and to support this podcast. And now here's my conversation with Russ Tedric. What is the most beautiful motion of.

An animal or robot that you've ever seen?

I think the most beautiful motion of a robot has to be the passive dynamic walkers. I think there's just something fundamentally beautiful, the ones in particular that Steve Collins built with Andy Rowena at Cornell, a 3D walking machine. So it was not confined to a boom or a plane that you put it on top of a small ramp, give it a little push. It's powered only by gravity. No controllers, no batteries whatsoever. It just falls down the ramp. And at the time it looked more natural, more graceful, more human like than any robot we'd seen to date. Powered only by gravity.

How does it work?

Well, okay, the simplest model, it's kind of like a Slinky. It's like an elaborate Slinky. One of the simplest models we used to think about it is actually a rimless wheel. So imagine taking a bicycle wheel but take the rim off. So it's now just got A bunch of spokes. If you give that a push, it still wants to roll down the ramp. But every time its foot, its spoke comes around and hits the ground, it loses a little energy. Every time it takes a step forward, it gains a little energy. Those things can come into perfect balance and actually they want to. It's a stable phenomenon. If it's going too slow, it'll speed up. If it's going too fast, it'll slow down and it comes into a stable periodic motion. Now you can take that rimless wheel, which doesn't look very much like a human walking, take all the extra spokes away, put a hinge in the middle. Now it's two legs. That's called our compass gate walker. That can still. You give it a little push, starts falling down a ramp. Looks a little bit more like walking. At least it's a biped. But what Steve and andy and Ted McGear started the whole exercise. But what Steve and Andy did was they took it to this beautiful conclusion where they built something that had knees, arms, a torso, the arms swung naturally. Give it a little push, and that looked like a stroll through the park.

How do you design something like that? I mean, is that art or science?

It's on the boundary. I think there's a science to getting close to the solution. I think there's certainly art in the way that they. They made a beautiful robot. But. But then the finesse because. Because this was work. They were working with a system that wasn't perfectly modeled, wasn't perfectly controlled. There's all these little tricks that you have to tune the suction cups at the knees, for instance, so that they stick, but then they release at just the right time. Or there's all these little tricks of the trade which really are art. But it was a point. I mean, it made the point. We were at that time, the walking robot. The best walking robot in the world was Honda's Asimo. Absolutely marvel of modern engineering.

This is 90s.

This was in 97 when they first released it, sort of announced P2 and then it went through. It was Asimo by then, in 2004.

And it looks like this very cautious.

Walking, like you're walking on hot coals or something like that.

I think it gets a bad rap. Asimo is a beautiful machine. It does walk with its knees bent. Our ATLAS walking had its knees bent. But actually Asimo was pretty fantastic, but it wasn't energy efficient. Neither was Atlas. When we worked on Atlas, none of our robots that have been that complicated have been Very energy efficient. But there's a thing that happens when you do control, when you try to control a system of that complexity, you try to use your motors to basically counteract gravity, take whatever the world's doing to you and push back. Erase the dynamics of the world and impose the dynamics you want, because you can make them simple and analyzable, mathematically simple. And this was a very sort of beautiful example that you don't have to do that. You can just let go, let physics do most of the work. Right. And you just have to give it a little bit of energy. This one only walked down a ramp. It would never walk on the flat. To walk on the flat, you have to give a little energy at some point. But maybe instead of trying to take the forces imparted to you by the world and replacing them, we should be doing is letting the world push us around and we go with the flow. Very Zen. Very Zen robot.

Yeah, but. Okay, so that sounds very Zen, but I can also imagine how many failed versions they had to go through. Like, how many. Like, I would say it's probably. Would you say it's in the thousands that they've had to have the system fall down before they figured out how to get.

I don't know if it's thousands, but it's a lot. It takes some patience, there's no question.

So in that sense, control might help a little bit.

Oh, the apps. I think everybody even at the time said that the answer is to do that with control. But it was just pointing out that maybe the way we're doing control right now isn't the way we should.

Got it.

So what about on the animal side.

The ones that figured out how to move efficiently? Is there anything you find inspiring or beautiful in the movement of any.

I do have a favorite example.

Okay.

So it sort of goes with the passive walking idea. So is there, you know, how. How energy efficient are animals? Okay. There's a great series of experiments by George Lauder at Harvard and Mike Tranafilo at mit. They were studying fish swimming in a water tunnel. Okay. And one of these, the type of fish they were studying were these rainbow trout. Because there was a phenomenon well understood that rainbow trout, when they're swimming upstream at mating season, they kind of hang out behind the rocks. And it looks like, I mean, that's tiring work. Swimming upstream, they're hanging out behind the rocks. Maybe there's something energetically interesting there. So they tried to recreate that. They put in this water tunnel a rock, basically a cylinder that had the same sort of vortex street, these eddies coming off the back of the rock that you would see in a stream. And they put a real fish behind this and watched how it swims. And the amazing thing is that if you watch from above what the fish swims when it's not behind a rock, it has a particular gait. You can identify the fish the same way you look at a human walking down the street. You sort of have a sense of how a human walks. The fish has a characteristic gait. You put that fish behind the rock, its gait changes. And what they saw was that it was actually resonating and kind of surfing between the vortices. Yeah. Now here was the experiment that really was the clincher because there was still, it wasn't clear how much of that was mechanics of the fish, how much of that is control the brain. So the clincher experiment, and maybe one of my favorites to date, although there are many good experiments, they took. They. This was now a dead fish. They took a dead fish, they put a string that went. That tied the mouse of the fish to the rock so it couldn't go back and get caught in the grates. And then they asked what would that dead fish do when it was hanging out behind the rock. And so what you'd expect, it sort of flopped around like a dead fish in the. In the vortex wake. Until something sort of amazing happens in this video that's worth putting in. Yeah. Right.

What happens?

The dead fish basically starts swimming upstream. Right. It's completely dead. No brain, no motors, no control. But it somehow the mechanics of the fish resonate with the vortex street and it starts swimming upstream. It's one of the best examples ever.

Who do you credit for that too? Is that just evolution? Constantly just figuring out by killing a lot of generations of animals, like the most efficient motion is that. Or maybe the physics of our world completely like, it's like evolution applied not only to animals, but just the entirety of it somehow drives to efficiency. Like nature likes efficiency. I don't know if that question even makes any sense.

I understand the question. That's reasonable. I mean, do they co. Evolve?

Yeah, somehow. Co. Yeah, Like I don't know if an environment can evolve.

But I mean, there are experiments that people do, careful experiments that show that animals can adapt to unusual situations and recover efficiency. So there seems like, at least in one direction, I think there is reason to believe that the animal's motor system and probably its mechanics adapt in order to be more efficient. But efficiency isn't the only Goal. Of course, sometimes it's too easy to think about only efficiency, but we have to do a lot of other things first, not get eaten, and then, all other things being equal, try to save energy.

By the way, let's draw a distinction between control and mechanics. Like, how can. How would you define each?

Yeah, I mean, I think part of the point is that we shouldn't draw a line as clearly as we tend to. But on a robot, we have motors and we have the links of the robot. Let's say if the motors are turned off, the robot has some passive dynamics. Okay. Gravity does the work. You can put springs. I would call that mechanics. Right. If we have springs and dampers, which our muscles are springs and dampers and tendons, but then you have something that's doing active work, putting energy in your motors. On the robot, the controller's job is to send commands to the motor that add new energy into the system. Right. So the mechanics and control interplay. Somewhere the divide is around, you know, did you decide to send some commands to your motor or did you just leave the motors off, let them do their work?

Would you say, is most of nature on the dynamic side or the control side? So, like, if you look at biological systems or if, you know, we're living in a pandemic now, like, do you think a virus is a. Do you think is a dynamic system or. Or is there a lot of control intelligence?

I think it's both. But I think we maybe have underestimated how important the dynamics are. Right. I mean, even our bodies, the mechanics of our bodies, certainly with exercise, they evolve. But so I actually, I lost a finger in early 2000s, and it's my fifth metacarpal. It turns out you use that a lot in ways you don't expect when you're opening jars. Even when I'm just walking around, if I bump it on something, there's a bone there that was used to taking contact. My fourth metacarpal wasn't used to taking contact. It used to hurt. Still does a little bit. But actually my bone has remodeled. Right. Over a couple years, the geometry, the mechanics of that bone changed to address the new circumstances. So the idea that somehow it's only our brain that's adapting or evolving is not right.

Maybe sticking on evolution for a bit because it's tended to create some interesting things of bipedal walking. Do you. Why the heck did evolution give us. I think we're. Are we the only mammals that walk on two feet?

No, I mean, there's a bunch of animals that do it a bit.

A bit.

There's a. I think we are the most successful bypass.

I think some. I think I read somewhere that the reason the, you know, evolution made us walk on two feet is because there's an advantage to being able to carry food back to the tribe or something like that. So like you can carry. It's kind of this communal cooperative thing, so like to carry stuff back to. To a place of shelter and so on to share with others. Do you understand at all the value of walking on two feet from both the robotics and a human perspective?

Yeah, there are some great books written about evolution of walking, evolution of the human body. I think it's easy though, to make bad evolutionary arguments.

Sure, most of them are probably bad, but what else can we do?

I mean, I think a lot of what dominated our evolution probably was not the things that worked well sort of in the steady state, you know, when things are. When things are good. But, but for instance, people talk about what we should eat now because our ancestors were meat eaters or, or whatever.

Oh yeah, I love that. Yeah.

But probably, you know, the reason that one pre, pre Homo sapien species versus another survived was not because of whether they ate well when there was lots of food, but when the ice age came, probably one of them happened to be in the wrong place. One of them happened to forage a food that was okay even when the glaciers came or something like that.

There's a million variables that contributed. And actually the amount of information we're working with in telling these stories, these evolutionary stories is, Is very little. So, yeah, just like you said, it seems like if we study history, it seems like history turns on like these little events that, that otherwise would seem meaningless. But in the grant, like when you in retrospect were turning points.

Absolutely.

And that, that's probably how like somebody got hit in the head with a rock because somebody slept with the wrong person back in the cave days and somebody get angry and that turned, you know, warring tribes combined with the environment, all those millions of things. And the meat eating, which I get a lot of criticism because I, I don't know. I don't know what your dietary processes are like, but these days I been eating only meat, which is. There's. There's a large community of people who say, yeah, probably make evolutionary arguments and say, you do a great job. There's probably an even larger community of people, including my mom, who says it's a deeply unhealthy, it's wrong, but I just feel good doing it. But you're right, these evolutionary arguments can be flawed. But is there anything interesting to pull out for?

There's a great book, by the way. Well, a series of books by Nicholas Taleb about Fooled by Randomness and Black Swan. Highly recommend them. But yeah, they make the point nicely that probably it was a few random events, that yes, maybe it was someone getting hit by a rock, as you say.

That said, do you think. I don't know how to ask this question or how to talk about this, but there's something elegant and beautiful about moving on two feet. Obviously biased because I'm human, but from a robotics perspective too. You work with robots on two feet. Is it all useful to build robots that are on two feet as opposed to four? Is there something useful about it the most?

I mean, the reason I spent a long time working on bipedal walking was because it was hard and it was. It challenged control theory in ways that I thought were important. I wouldn't have ever tried to convince you that you should start a company around bipeds or something like this. There are people that make pretty compelling arguments, right? I think the most compelling one is that the world is built for the human form. And if you want a robot to work in the world we have today, then having a human form is a pretty good way to go. There are places that a biped can go that would be hard for other form factors to go, even natural places. But, you know, at some point in the long run, we'll be building our environments for our robots, probably. And so maybe that argument falls aside.

So you famously run barefoot. Do you still run barefoot?

I still run barefoot.

That's so awesome.

Much to my wife's chagrin.

Do you want to make an evolutionary argument for why running barefoot is advantageous? What have you learned about human and robot movement in general from running barefoot? Human or robot and. Or.

Well, you know, it happened the other way. Right. So I was studying walking robots and I was. There's a great conference called the Dynamic Walking Conference where it brings together both the biomechanics community and the walking robots community. And so I had been going to this for years and hearing talks by people who study barefoot running and other. The mechanics of running. So I, I did eventually read Born to Run. Most people read Born to Run in the. Right. The other thing I had going for me is actually that I, I wouldn't. I wasn't a runner before and I learned to run after I had learned about barefoot running or I mean, started running longer distances. So I didn't have to unlearn. And I'm definitely, I'm a big fan of it for me, but I'm not gonna, I tend to not try to convince other people. There's people who run beautifully with shoes on and that's good. But here's why it makes sense. For me, it's all about the long term game, right? So I think it's just too easy to run 10 miles, feel pretty good. And then you get home at night and you realize, my knees hurt. I did something wrong or something. Right? If you take your shoes off, then if you hit hard with your foot at all, then it hurts. You don't like run 10 miles and then realize you've done some damage. You have immediate feedback telling you that you've done something that's maybe suboptimal and you change your gait. I mean, it's even subconscious if I, right now, having run many miles barefoot, if I, I put a shoe on my gate changes in a way that I think is not as good. Um, so, so it, it makes me land softer. And I think my, my goals for running are to do it for as long as I can into old age, not to win any races. And so for me, this is a, you know, a way to protect myself.

Yeah, I think first of all, I've tried running barefoot many years ago, probably the other way. Just, just, just reading Born to run, but just to understand because I felt like I couldn't put in the miles that I wanted to. And it feels like running for me and I think for a lot of people was one of those activities that we do often and we never really try to learn to do correctly. Like, it's funny, there's so many activities we do every day, like brushing our teeth. Right. I think a lot of us, at least me, probably have never deeply studied how to properly brush my teeth. Right. Or wash as now with a pandemic or how to properly wash our hands. We do it every day, but we haven't really studied. Like, am I doing this correctly? But running felt like one of those things. It was absurd not to study how to do correctly because it's the source of so much pain and suffering. Like, I hate running, but I do it. I do it because I hate it, but it, I feel good afterwards, but I think it feels like you need to learn how to do it properly. So that's where barefoot running came in. And then I quickly realized that my gait was completely wrong. I was taking huge like steps and landing hard on the heel. All those elements and so, yeah, from that, I actually learned to take really small steps. Look, I already forgot the number, but I feel like it was 180aminute or something like that. And I remember I actually just took songs that are 180 beats per minute and then, like, tried to run at that beat just to teach myself. It took a long time. And I feel like after a while, you learn to run, you adjust it properly without going all the way to barefoot. But I feel like barefoot is the legit way to do it. I mean, I think a lot of people would be really curious about it. Can you. If they're interested in trying, what would you. How would you recommend they start or try or explore slowly?

That's the biggest thing people do is they are excellent runners and they're used to running long distances or running fast, and they take their shoes off and they hurt themselves instantly trying to do something that they were used to doing. I think I lucked out in the sense that I. I couldn't run very far when I first started trying. And I run with minimal shoes, too. I mean, I will, you know, bring along a pair of actually, like, aqua socks or something like this I can just slip on, or running sandals. I've tried all of them.

What's the difference between a minimal shoe and nothing at all? What's like, feeling wise? What does it feel like?

There is a. I mean, I noticed my gait changing, right? So, I mean, your. Your foot has as many muscles and sensors as your hand does, right?

Sensors.

Ooh. Okay.

And we do amazing things with our hands, and we stick our foot in a big, solid shoe. Right? So there's. I think, you know, when you're barefoot, you're just giving yourself more proprioception, and that's why you're more aware of some of the gate flaws and stuff like this. Now you have less protection, too. So rocks and stuff. I mean. Yeah, so. So I think people are. Who are afraid of barefoot running are worried about getting cuts or getting stepping on rocks. First of all, even if that was a concern, I think those are all, like, very short term. You know, if I get a scratch or something, it'll heal in a week. If I blow out my knees, I'm done running forever. So I will trade the short term for the long term anytime. But even then, you know, this again, to my wife's chagrin, your feet get tough, right? And callous.

Okay.

Yeah, I can run over almost anything now.

I mean, what maybe can you talk about? Is there hint, like, is there tips or tricks that you have suggestions about, like, if I wanted to try it.

You know, there is a good book, actually. Uh, there's probably more good books since I read them, but Ken Bob. Barefoot Ken Bob Saxton. Um, he's an interesting guy, but I think his book captured the right way to describe running, barefoot running to somebody better than any other I've seen.

So you run pretty good distances and you bike. And is. Is there.

You know, if we talk about bucket.

List items, is there something crazy on your bucket list, athletically, that you hope to do one day?

I mean, my commute is already a little crazy.

What are we talking about here? What distance are we talking about?

Well, I live about 12 miles from MIT, but you can find lots of different ways to get there. So, I mean, I've run there for many years. I biked there always. Yeah, but normally I would try to run in and then bike home. Bike in, run home.

But you have run there and back before?

Sure.

Barefoot?

Yeah, yeah. Or with minimal shoes or whatever.

That. 12. 12 times two.

Yeah.

Okay.

It became kind of a game of how can I get to work? I've rollerbladed. I've done all kinds of weird stuff, but my favorite one these days is I've been taking the Charles river to work so I can put in a little rowboat not so far from my house. But the Charles river takes a long way to get the mit, so I can spend a long time getting there. And it's, you know, it's not about. I don't know, it's just about. I've had people ask me, how can you justify taking that time? But for me, it's just a magical time to think, to compress, decompress, you know, especially. I'll wake up, do a lot of work in the morning, and then I kind of have to just let that settle before I'm ready for all my meetings. And then on the way home, it's a great time to sort of let that settle.

So you. You lead a. Like a. A large group of people. I mean, you're. Is there days where you're like, oh, shit, I gotta get to work in an hour? Like, I mean, uh, is. Is there. Is there a tension there? Where. And like, if we look at the grand scheme of things, just like you said, long term, that meeting probably doesn't matter. Like, you can always say, I'll just. I'll run and let the meeting happen how it happens. Like, what. How do you. That Zen. How do you. What do you do with that tension between the real world Saying urgently you need to be there. This is important. Everything is melting down. How we're going to fix this robot. There's this critical meeting and then there's this design beauty of just running. The simplicity of it. You along with nature. What do you do with that?

I would say I'm not a fast runner particularly probably my fastest splits ever was when I had to get to daycare on time because they were going to charge me, you know some, some dollar per minute that I was late. I've run some fast splits to daycare. Uh, but that those times are past now. Um, I think work, you can find a work life balance in that way. I think you just have to um. I think I am better at work because I take time to think on the way in. So I plan my day around it. Um, and I rarely feel that those are really in at odds.

So what the bucket list item.

If we're talking 12 times two, we're approaching a marathon. Have you run an ultra marathon before? Do you do races? Is there.

What's to win?

Not to.

I'm not going to take a dinghy across the Atlantic or something if that's what you want. But if someone does and wants to write a book I would totally read it because I'm a sucker for that kind of thing. No, I do have some fun things that I will try. You know I like to. When I travel I almost always bike to Logan airport and fold up a little folding bike and then take it with me and bike to wherever I'm going and it's taken me or I'll take a stand up paddle board these days on the airplane and then I'll try to paddle around where I'm going or whatever. And I've done some crazy things but.

But not for the. You know, I now talk. I don't know if you know who David Goggins is by any chance that well but. Yeah, but I, I talk to him now every day. So he's the person who made me do this stupid challenge. So he, he's insane. And he does things for the purpose in, in the best kind of way. He does things like for the explicit purpose of suffering. Like he picks the thing that like whatever he thinks he can do, he does more. So is that. Do you have that thing in you or are you.

I think it's become the opposite.

It's so you're like that dynamical system that the walker, the efficient.

Yeah, it's leave no pain. Right. You should end feeling better than you started.

Okay.

But it's mostly, I think, and Covid has tested this because I've lost my commute. I think I'm perfectly happy walking around, around town with my wife and kids, if they could get them to go. And it's more about just getting outside and getting away from the keyboard for some time, just to let things compress.

Let's go into robotics a little bit. What to use the most beautiful idea in robotics, whether we're talking about control or whether we're talking about optimization and the math side of things, or the engineering side of things, or the philosophical side of things.

I think I've been lucky to experience something that not so many roboticists have experienced, which is to hang out with some really amazing control theorists. And the clarity of thought that some of the more mathematical control theory can bring to even very complex, messy looking problems is really, it really had a big impact on me. And I had a day, even just a couple weeks ago where I had spent the day on a zoom robotics conference having great conversations with lots of people, felt really good about the ideas that were flowing and the like. And then I had a, you know, late afternoon meeting with one of my favorite control theorists. And we went from these abstract discussions about maybes and what ifs and what a great idea to these super precise statements about systems that aren't that much more simple or abstract than the ones I care about deeply. And the contrast of that is, I don't know, it really gets me. I think people underestimate maybe the power of clear thinking. And so, for instance, deep learning is amazing. I use it heavily in our work. I think it's changed the world. Unquestionable. It makes it easy to get things to work without thinking as critically about it. So I think one of the challenges as an educator is to think about how do we make sure people get a taste of the more rigorous thinking that I think goes along with some different approaches.

Yeah, so that's really interesting. So understanding the fundamentals, the first principles of the, of the problem, where in this case it's mechanics, like how a thing moves, how a thing behaves, like all the forces involved, like really getting a deep understanding of that, I mean from physics, the first principle thing come from physics. And here it's literally physics. Yeah. And this applies in deep learning. This applies to not just, I mean it applies so cleanly in robotics, but it also applies to just in any data set. I find this true. I mean, driving as well. There's a lot of folks in it that work on autonomous vehicles that don't study driving, like deeply. I, I might be coming a little bit from the psychology side, but I remember I spent a ridiculous number of hours at lunch at this like lawn chair. And I would sit somewhere, somewhere in MIT's campus, there's a few interesting intersections and we just watch people cross. So we were studying pedestrian behavior. And I felt like as you record a lot of video to try. And then there's the computer vision, extracts their movement, how they move their head and so on. But like, every time I felt like I didn't understand enough, I, I just, I felt like I wasn't understanding. What, how are people signaling to each other? What are they thinking? How cognizant are they of their fear of death? Like, what are we? Like, what's the game? What's the underlying game theory here? What are, what are the, the, the incentives? And then I finally found a live stream of an intersection that's like high def that I just, I would watch so I wouldn't have to sit out there. But that's interesting. So like I, that's tough.

That's a tough example. I mean, the learning humans are involved not just because human, but I think the learning mantra is that basically the statistics of the data will tell me things I need to know. Right. And for the example you gave of all the nuances of eye contact or hand gestures or whatever that are happening for these subtle interactions between pedestrians and traffic, maybe the data will tell that story. I may be even one level more meta than what you're saying for a particular problem. I think it might be the case that data should tell us the story. But I think there's a rigorous thinking that is just an essential skill for a mathematician or an engineer that I just don't want to lose it. There are certainly super rigorous methods, rigorous control or, sorry, machine learning people. I just think deep learning makes it so easy to do some things that our next generation are not immediately rewarded for going through some of the more rigorous approaches. And then I wonder where that takes us. Well, I'm actually optimistic about it. I just want to do my part to try to steer that rigorous thinking.

So there's like two questions I want to ask. Do you have sort of a good example of rigorous thinking where it's easy to get lazy and not do the rigorous thinking? And the other question I have is like, do you have advice of how to practice rigorous thinking in all the computer science disciplines that, that we've mentioned?

Yeah, I mean, there are times where problems that can be solved with well known Mature methods could also be solved with a deep learning approach. And there's an argument that you must use learning even for the parts we already think we know, because if the human has touched it, then you've biased the system and you've suddenly put a bottleneck in there that is your own mental model. But something like inverting a matrix, I think we know how to do that pretty well, even if it's a pretty big matrix and we understand that pretty well. And you could train a deep network to do it, but you shouldn't, probably.

So in that sense, rigorous thinking is understanding the scope and the limitations of the methods that we have, like how to use the tools of mathematics properly.

Yeah, I think, you know, taking a class on analysis is all I'm sort of arguing is to take, take a chance to stop and, and force yourself to think rigorously about even, you know, the rational numbers or something. You know, it doesn't have to be the end all problem. But that exercise of clear thinking, I think, goes a long way. And I just want to make sure we, we keep preaching, don't lose it.

Yeah, but do you think when you're.

Doing like, rigorous thinking or like maybe trying to write down equations or to sort of explicitly like formally describe a system, do you think we naturally simplify things too much? Is that a danger you run into, like, in order to be able to understand something about the system mathematically, we make it too much of a toy example.

But I think that's the good stuff, right?

That's how you understand the fundamentals.

I think so. I think maybe even that's a key to intelligence or something. But I mean, okay, what if Newton and Galileo had deep learning and they had done a bunch of experiments and they told the world, here's your weights of your neural network. We've solved the problem, you know, where would we be today? I don't, I don't think we'd be as far as we. As we are. There's something to be said about having the simplest explanation for a phenomenon. So I don't doubt that we can train neural networks to predict even physical, you know, F equals MA type equations. But I may be. I want another Newton to come along because I think there's more to do in terms of coming up with the simple models for more complicated tasks.

Yeah, let's not offend AI systems from 50 years from now that are listening to this that are probably better at, might be better at coming up with F equals MA equations themselves.

Sorry. I actually think learning is probably A route to achieving this. But the representation matters. And I think having a function that takes my inputs to outputs that is arbitrarily complex may not be the end goal. I think there's still the most simple or parsimonious explanation for the data. Simple doesn't mean low dimensional. That's one thing. I think that we've a lesson that we've learned so, you know, a standard way to do model reduction or system identification and controls is to. The typical formulation is that you try to find the minimal state dimension realization of a system that hits some error bounds or something like that. And that's maybe not. I think we're learning that state dimension is not the right metric of complexity, of complexity. But for me, I think a lot about contact, the mechanics of contact. If a robot hand is picking up an object or something. And when I write down the equations of motion for that, they're. They look incredibly complex. Not because, actually not so much because of the dynamics of the hand when it's moving, but it's just the interactions and when they turn on and off. Right. So having a high dimensional, you know, but simple description of what's happening out here is fine.

But.

But if when I actually start touching, if I write down a different dynamical system for every polygon on my robot hand and every polygon on the object, whether it's in contact or not, with all the combinatorics that explodes there, then that's too complex. So I need to somehow summarize that with a more intuitive physics way of thinking. And yeah, I'm very optimistic that machine learning will get us there.

First of all. I mean, I'll probably do it in the introduction, but you're one of the great robotics people at mit. You're a professor at mit, you've teach a lot of amazing courses, you run a large group and you have a important history for mit, I think as being a part of the DARPA Robotics challenge. Can you maybe first say what is the DARPA Robotics challenge? And then tell your story around it, your journey with it?

Yeah, sure. So the DARPA Robotics Challenge, it came on the tales of the DARPA Grand Challenge and DARPA Urban Challenge, which were the challenges that brought us put a spotlight on self driving cars. Gil Pratt was at DARPA and pitched a new challenge that involved disaster response. It didn't explicitly require humanoids, although humanoids came into the picture. This happened shortly after the Fukushima disaster in Japan. And our challenge was motivated roughly by that, because that was a case where if we had had Robots that were ready to be sent in, there's a chance that we could have averted disaster. And certainly after, in the disaster response, there were times where we would have loved to have sent robots in. So in practice, what we ended up with was a grand challenge, a DARPA robotics challenge where Boston Dynamics was, was to make humanoid robots. People like me and the amazing team at MIT were competing first in a simulation challenge to try to be one of the ones that wins the right to work on one of the Boston Dynamics humanoids in order to compete in the final challenge, which was a physical challenge.

And at that point it was already.

So it was decided as humanoid robots.

Early on there were two tracks. You could enter as a hardware team where you brought your own robot, or you could enter through the virtual robotics challenge as a software team that would try to win the right to use one of the Boston Dynamics robots, which are called Atlas.

Atlas humanoid robots.

It was a 400 pound marvel, but pretty big, scary looking robot.

Expensive, too expensive at the time, yeah. Okay, so I mean, how did you feel at the prospect of this kind of challenge? I mean, it seems, you know, autonomous vehicles. Yeah, I guess that sounds hard, but not really from a robotics perspective. It's like, didn't they do it in the 80s is the kind of feeling I would have. Like when you first look at the problem, it's on wheels, but like humanoid robots, that sounds really hard. So what, like, what are the. Psychologically speaking, what were you feeling? Excited, scared? Why the heck did you get yourself involved in this kind of messy challenge?

We didn't really know for sure what we were signing up for in the sense that you could have something that, as it was described in the call for participation, that could have put a huge emphasis on the dynamics of walking and not falling down and walking over rough terrain, or the same description, because the robot had to go into this disaster area and turn valves and pick up a drill, cut the hole through a wall. It had to do some interesting things. The challenge could have really highlighted perception and autonomous planning, or it ended up that, you know, locomoting over complex terrain played a pretty big role in the competition. So.

And the degree of autonomy wasn't clear.

The decree of autonomy was always a central part of the discussion. So what wasn't clear was how we would be able, how far we'd be able to get with it. So the idea was always that you want semi autonomy, that you want the robot to have enough compute that you can have a degraded network link to a human. And so the Same way we had degraded networks at many natural disasters. You'd send your robot in, you'd be able to get a few bits back and forth, but you don't get to have enough potentially to fully operate the robot. Every joint of the robot. So, and then the question was, and the gamesmanship of the organizers was to figure out what we're capable of, push us as far as we could so that it would differentiate the teams that put more autonomy on the robot and had a few clicks and just said, go there, do this, go there, do this, versus someone who's picking every footstep or something like that.

So what were some memories? Painful, triumphant, from the experience, like what was that journey? Maybe if you can dig in a little deeper, maybe even on the technical side, on the team side, that whole process of, from the early idea stages to actually competing, I mean, this was.

A defining experience for me. It came at the right time for me in my career. I had gotten tenure before I was due a sabbatical. And most people do something relaxing and restorative for a sabbatical.

So you got tenure before this?

Yeah, yeah, yeah. It was a good time for me. I had, I had, we had a bunch of algorithms that we were very happy with. We wanted to see how far we could push them. And this was a chance to really test our metal to do more proper software engineering. The team, we all just worked our butts off. We, you know, we're in that lab almost all the time. Okay. So, I mean there were some, of course, high highs and low lows throughout that. Anytime you're, you know, not sleeping and devoting your life to a 400 pound humanoid. I remember actually one funny moment where we're all super tired and so ATLAS had to walk across cinder blocks. That was one of the obstacles. And I remember ATLAS was powered down, hanging limp, you know, on the, on its harness. And the humans were there like laying, you know, picking up and laying the brick down so that the robot could walk over it. And I thought, what is wrong with this? You know, we've got a robot just watching us do all the manual labor so that it can take its little stroll across the terrain. But I mean, even the, even the virtual robotics challenge was super nerve wracking and dramatic. I remember. So we were using Gazebo as a simulator on the cloud and there was all these interesting challenges. I think the investment that OSR fc, whatever they were called at that time, Brian Gerke's team at Open Source Robotics, they were pushing on the capabilities of Gazebo in order to scale it to the complexity of these challenges. So, you know, up to the virtual competition. So the virtual competition was you will sign on at a certain time and we'll have a network connection to another machine on the cloud that is running the simulator of your robot. And your controller will run on this computer and the physics will run on the other. And you have to connect. Now, the physics, they wanted it to run at real time rates because there was an element of human interaction and humans could. If you do want a teleop, it works way better if it's at frame rate.

Oh, cool.

But it was very hard to simulate these complex scenes at real time rate. So right up to like days before the competition, the simulator wasn't quite at real time rate. And that was great for me because my controller was solving a pretty big optimization problem and it wasn't quite at real time rate. So I was fine. I was keeping up with the simulator. We were both running at about 0.7. And I remember getting this email. And by the way, the perception folks on our team hated that. They knew that if my controller was too slow, the robot was going to fall down. And no matter how good their perception system was, if I can't make my controller fast enough anyways, we get this email like three days before the virtual competition. It's for all the marbles. We're going to either get a humanoid robot or we're not. And we get an email saying, good news, we made the robot does the simulator faster. It's now one point. And I was just like, oh man, what are we going to do here? So that came in late at night for me.

A few days ahead.

A few days ahead. I went over. It happened at Frank Permenter, who's a very, very sharp. He was a student at the time working on optimization. He was still in lab. Frank, we need to make this quadratic programming solver faster. Not like a little faster. It's actually. And we wrote a new solver for that QP together that night.

And you solved it.

So there's a really hard optimization problem that you're constantly solving. You didn't make the optimization problem simpler, you wrote a new solver.

So I mean, your observation is almost spot on. What we did was what everybody. I mean, people know how to do this, but we had not yet done this idea of warm starting. So we are solving a big optimization problem at every time step. But if you're running fast enough, the optimization problem you're solving on the last time step is pretty similar. To the optimization you're going to solve with the next. We had course had told our commercial solver to use warm starting, but even the interface to that commercial solver was causing us these delays. So what we did was we basically wrote, we called it FastQP at the time. And we wrote a very lightweight, very fast layer which would basically check if nearby solutions to the quadratic program, which were very easily checked, could stabilize the robot. And if they couldn't, we would fall back to the solver.

You couldn't really test this well. Right.

So we always knew that if we fell back to. If we, it got to the point where if for some reason things slowed down and we fell back to the original solver, the robot would actually literally fall down. Um, so it was, it was a harrowing sort of edge we were ledge. We were sort of on. But I mean actually like the, the 400 pound humanoid could come crashing to the ground if you, if you, if your solver's not fast enough. But yeah, we had lots of good experiences.

So. Can I ask you a weird question? I get about the idea of hard work.

So actually people like students of yours.

That I've interacted with and just. And robotics people in general, but they, they have moments at moments have worked harder than most people I know in terms of. If you look at different disciplines of how hard people work. But, but they're also like the happiest, like, just like, I don't know. It's the same thing with like running people that push themselves to like the limit. They also seem to be like the most like full of life somehow. And I get often criticized, like, you know, you're not getting enough sleep, what are you doing to your body? Blah, blah, blah, like this kind of stuff. And I usually just kind of respond like, I'm doing what I love, I'm passionate about. I love it. I feel like it's, it's invigorating. I actually think, I don't think the lack of sleep is what hurts you. I think what hurts you is stress and lack of doing things that you're passionate about. But in this world, yeah, I mean, can you comment about why the heck robotics people are willing to push themselves to that degree? Is there value in that? And why are they so happy?

I think you got it right. I mean, I think the causality is not that we work hard, and I think other disciplines work very hard too, but I don't think it's that we work hard and therefore we are happy. I think we found something that we're truly passionate about it, makes us very happy, and then we get a little involved with it and spend a lot of time on it. What a luxury to have something that you want to spend all your time on. Right.

We could talk about this for many hours, but maybe if we could pick. Is there something on the technical side on the approach you took that's interesting that turned out to be a terrible failure or a success that you carry into your work today about all the different ideas that were involved in making, whether in the, in the simulation or in the, in the real world, making this semi autonomous system work.

I mean, it really did teach me something fundamental about what it's going to take to get robustness out of a system of this complexity. I would say the DARPA challenge really was foundational in my thinking. I think the autonomous driving community thinks about this. I think lots of people thinking about safety critical systems that might have machine learning in the loop are thinking about these questions. For me, the DARPA challenge was the moment where I realized, you know, we've spent every waking minute running this robot and again in for the physical competition. Days before the competition, we saw the robot fall down in a way it had never fallen down before. I thought, you know, how could we have found that, you know, we only have one robot, it's running almost all the time. We just didn't have enough hours in the day to test that robot. Something has to change, right? And I think that, I mean, I would say that the team that won was, was from kaist, was the team that had two robots and was able to do not only incredible engineering, just absolutely top rate engineering, but also they were able to test at a rate and discipline that we didn't keep up with.

What does testing look like? What are we talking about here? Like, what's, what's a loop of test like from start to finish? What does a loop of testing look like?

Yeah, I mean, I think there's a whole philosophy to testing. There's the unit tests and you can do that on a hardware. You can do that in a small piece of code. You write one function, you should write a test that checks that function's input and outputs. You should also write an integration test. At the other extreme of running the whole system together, you know, where that try to turn on all the different functions that you think are correct. It's much harder to write the specifications for a system level test, especially if that system is as complicated as a humanoid robot. But the philosophy is sort of the same on the real Robot, it's no different. But on a real robot, it's impossible to run the same experiment twice. So if you see a failure, you hope you caught something in the logs that tell you what happened. But you'd probably never be able to run exactly that experiment again. And right now I think our philosophy is just basically Monte Carlo estimation is just run as many experiments as we can. Maybe try to set up the environment to make the things we are worried about happen as often as possible. But really we're relying on somewhat random search in order to test. Maybe that's all we'll ever be able to. But I think, you know, because there's an argument that the things that'll get you are the things that are really nuanced in the world. And it'd be very hard to, for instance, put back in a simulation.

Yeah, the, I guess the edge cases.

What was the hardest thing like, so you said walking over rough terrain, like just taking footsteps.

I mean people, it's so dramatic and.

Painful in a certain kind of way to watch these videos from the DRC of robots falling.

Yep.

It's just so heartbreaking. I don't know, maybe it's because for me at least, we anthropomorphize the robot.

Of course, every one of those funny.

For some reason, like humans falling is funny for. I don't. It's some dark reason. I'm not sure why it is so. But it's also like tragic and painful and so speaking of which, I mean, what made the robots fall and fail in your view?

So I can tell you exactly what happened on a. I contributed one of those. Our team contributed one of those spectacular falls. Every one of those falls has a complicated story. I mean, at one time the power effectively went out on the robot because it had been sitting at the door waiting for a green light to be able to proceed and its batteries and therefore it just fell backwards and smashed its head to the ground. And it was hilarious. But it wasn't because of bad software. Right. But for ours. So the hardest part of the challenge, the hardest task in my view was getting out of the Polaris. It was actually relatively easy to drive the Polaris.

Can you tell the story? Sorry to interrupt the story of the car. People should watch this video. I mean the thing you've come up with is just brilliant. But anyway, sorry, we kind of joke.

We call it the big robot little car problem because somehow the race organizers decided to give us a 400 pound humanoid and they also provided the vehicle, which was a little Polaris and the robot didn't really fit in the car. So you couldn't drive the car with your feet under the steering column. We actually had to straddle the main column and have basically one foot in the passenger seat, one foot in the driver's seat, and then drive with our left hand. But the hard part was we had to then park the car, get out of the car. It didn't have a door that was okay, but it's just getting up from crouched, from sitting when you're in this very constrained environment.

First of all, I remember after watching those videos, I was much more cognizant of how hard it is for me to get in and out of the car and out of the car especially like it's actually a really difficult control problem. Yeah, I'm very cognizant of it when I'm like injured for whatever reason.

It's really hard.

Yeah. So how did you approach.

You think of NASA's operations and they have these checklists, pre launch checklists and the like. We weren't far off from that. We had this big checklist and on the first day of the competition we were running down our checklist and one of the things we had to do, we had to turn off the controller, the piece of software that was running that would drive the left foot of the robot in order to accelerate on the gas. And then we turned on our balancing controller and the nerves jitters of the first day of the competition, someone forgot to check that box and turn that controller off. So we used a lot of motion planning to figure out a sort of configuration of the robot that we could get up and, and over. We relied heavily on our balancing controller and, and basically there was. When the robot was in one of its most precarious, you know, sort of configurations, trying to sneak its big leg out of the, out of the side. The other controller that thought it was still driving told its left foot to go like this and, and that wasn't good, but, but it turned disastrous for us because what happened was a little bit of push here actually. If we have videos of us, you know, running into the robot with a 10 foot pole and it kind of will recover, but this is a case where there's no space to recover. So a lot of our secondary balancing mechanisms about like take a step to recover. They were all disabled because we were in the car, there's no place to step. So we were relying on our just lowest level reflexes. And even then, I think just hitting the foot on the seat, on the floor we probably could have recovered from it. But the thing that was bad that happened is when we did that and we jostled a little bit, the tailbone of our robot was only a little off the seat. It hit the seat and the other foot came off the ground just a little bit. And nothing in our plans had ever told us what to do. If your butt's on the seat and your feet are in the air.

Feet in the air.

And then the thing is, once you get off the script, things can go very wrong. Because even our state estimation, our system that was trying to collect all the data from the sensors and understand what's happening with the robot, it didn't know about this situation. So it was predicting things that were just wrong. And then we did a violent shake and fell off in our face first.

On out of the robot, but like into the destination.

That's true. We fell in. We got our point.

But so is there any hope for. That's interesting. Is there any hope for ATLAS to be able to do something when it's just on its butt and feet in the air?

Absolutely.

So you can. What do you.

No. So that's, that is one of the big challenges and I think it's still true. You know, Boston Dynamics and Animal and there's this incredible work on legged robots happening around the world. Most of them still are very good at the case where you're making contact with the world at your feet and they have typically point feet, relatively balls on their feet. For instance, if those robots get in a situation where the elbow hits the wall or something like this, that's a pretty different situation. Now they have layers of mechanisms that will make, I think the more mature solutions have ways in which the controller won't do stupid things. But a human, for instance, is able to leverage incidental contact in order to accomplish a goal. In fact, I might, if you push me, I might actually put my hand out and make a new, brand new contact. The feet of the robot are doing this on quadrupeds. But we mostly in robotics are afraid of contact on the rest of our body, which is crazy. There's this whole field of motion planning, collision free motion planning, and we write very complex algorithms so that the robot can dance around and make sure it doesn't touch the world.

So people are just afraid of contact because contact the scene is a difficult.

It'S still a difficult control problem and sensing problem.

Now you're a serious person. I'm a little bit of an idiot and I'm going to ask you some dumb questions, so I do.