Grant Sanderson: Math, Manim, Neural Networks & Teaching with 3Blue1Brown | Lex Fridman Podcast #118

The following is a conversation with Grant Sanderson, his second time on the podcast. He's known to millions of people as the mind behind 3blue1brown, a YouTube channel where he educates and inspires the world with the beauty and power of mathematics. Quick summary of the sponsors Dollar Shave Club, DoorDash and Cash App. Click the sponsor links in the description to get a discount and to support this podcast, especially for the two new sponsors, Dollar Shave Club and DoorDash. Let me say as a side note, I think that this pandemic challenged millions of educators to rethink how they teach, to rethink the nature of education. As people know, Grant is a master elucidator of mathematical concepts that may otherwise seem difficult or out of reach for students and curious minds. But he's also an inspiration to teachers, researchers and people who just enjoy sharing knowledge like me. For what it's worth, it's one thing to give a semester's worth of multi hour lectures. It's another to extract from those lectures the most important, interesting, beautiful and difficult concepts and present them in a way that makes everything fall into place. That is the challenge that is worth taking on. My dream is to see more and more of my colleagues at MIT and world experts across the world summon their inner three Blue, one Brown and create the canonical explainer videos on a topic that that they know more than almost anyone else in the world. Amidst the political division, the economic pain, the psychological medical toll of the virus, masterfully crafted educational content feels like one of the beacons of hope that we can hold onto. If you enjoy this thing, subscribe on YouTube, review it with five stars on Apple Podcast, follow on Spotify, support on Patreon, or connect with me on Twitter lexfriedman. Of course, after you go immediately, which you already probably have done a long time ago, and subscribe to 3blue1brown YouTube.

Channel, you will not regret it.

As usual, I'll do a few minutes of AS now and no ads in the middle. I try to make these interesting, but I give you time stamps so you can skip. But still, please do check out the sponsors by clicking the links in the description, especially the two new ones, DoorDash and Dollar Shave Club. They're evaluating us looking at how many people go to their site and get their stuff in order to determine if they want to support us for the long term. So you know what to do. It's the best way to support this podcast. As always, this show is sponsored by Dollar Shave Club. Try them out with a one time offer for only $5 in free shipping at dollarshaveclub.com Lex starter kit comes with a six blade razor, refills and all kinds of other stuff that makes shaving feel great. I've been a member of Dollar Shave Club for over five years now and actually signed up when I first heard about them on the Joe Rogan podcast. And now we have come full circle. I feel like I've made it now that I can do a read for them just like Joe did all those years ago. For the most part I've just used the razor and the refills, but they encouraged me to try the shave butter, which I've never used before. So I did and I love it. Not sure how the chemistry of it works out, but it's translucent somehow, which is a cool new experience. Again, try the Ultimate Shave Starter Set today for just 5 bucks plus free shipping@dollarshaveclub.com Lex this show is also sponsored by DoorDash. Get 5 bucks off and zero delivery fees on your first order of $15 or more when you download the DoorDash app and enter code Lex. I have so many memories of working late nights for a deadline with a team of engineers and eventually taking a break to argue about which doordash restaurant to order from and when the food came. Those moments of bonding, of exchanging ideas, of pausing to shift attention from the programs to the humans were special. These days, for a bit of time, I'm on my own, sadly, so I miss that camaraderie. But actually, DoorDash is still there for me. There's a million options that fit into.

My keto diet ways.

Also, it's a great way to support restaurants in these challenging times. Once Again, download the DoorDash app and enter code LEX to get five bucks off and zero delivery fees on your first order of $15 or more. Finally, this show is presented by Cash app, the 1 finance app in the App Store. When you get it, use code lexpodcast. Cash App lets you send money to friends, buy bitcoin, and invest in the stock market with as little as $1. It's one of the best design interfaces of an app that I've ever used. To me, good design is when everything is easy and natural. Bad design is when the app gets in the way, either because it's buggy or because it tries too hard to be helpful. I'm looking at you, Clippy. Anyway, there's a big part of my brain and heart that love to design things and also to appreciate great design by others. So again, if you get Cash App from the App Store, Google Play and use code LEXPodcast. You get $10. And Cash App will also donate $10 to FIRST, an organization that is helping to advance robotics and STEM education for young people around the world. And now here's my conversation with Grant Sanderson.

You've spoken about Richard Feynman as someone you admire.

I think last time we spoke, we.

Ran out of time. So I wanted to talk to you about him. Who is Richard Feynman to you in your eyes? What impact did he have on you?

I mean, I think a ton of people like Feynman. He's probably. It's a little bit cliche to say.

That you like Feynman, right?

That's almost like when you don't know.

What to say about sports and you just point to the super bowl or something as something you enjoy watching.

But I do actually think there's a layer to Feynman that sits behind the iconography. One thing that just really struck me was this letter that he wrote to his wife two years after she died. So during the Manhattan Project, she had polio.

Tragically, she died. They were just young, madly in love.

And the icon of Feynman is this almost this mildly sexist, womanizing philanderer, at.

Least on the personal side.

But you read this letter, and I.

Can try to pull it up for you if I want.

And it's just this absolutely heartfelt letter.

To his wife saying how much he loves her even though she's dead, and kind of what she means to him, how no woman can ever measure up to her.

And it shows you that the Feynman.

That we've all seen in Surely you're joking is different from the Feynman in reality.

And I think the same kind of goes in his science, where he kind of sometimes has this output of being this aw, shucks character like everyone else is coming in.

There's these fancy falutin formulas, but I'm.

Just going to try to whittle it.

Down to its essentials, which is so appealing because we love to see that kind of thing.

But when you get into it, what he was doing was actually quite deep, very much mathematical. That should go without saying, but I remember reading a book about Feynman in a cafe once, and this woman looked at me and saw that it was about Feynman. She was like, oh, I love him. I read Julia, you're joking. And she started explaining to me how he was never really a math person. And I don't understand how that can possibly be a public perception about any physicist. But for whatever reason that worked into his aura that he sort of shooed.

Off math in place of true science.

The reality of it is he was deeply in love with math and was.

Much more going in that direction and had a clicking point into seeing that.

Physics was a way to realize that.

And all the creativity that he could output in that direction was instead poured towards things like fundamental. Not even fundamental theories, just emergent phenomena and everything like that.

So to answer your actual question, what I like about his way of going at things is, is this constant desire.

To reinvent it for himself. Like when he would consume papers, the way he'd describe it, he would start to see what problem he was trying to solve and then just try to solve it himself to get a sense of personal ownership and then from there see what others had done.

Is that how you see problems yourself?

That's actually an interesting point. When you first are inspired by a certain idea that you maybe want to teach or visualize or just explore on your own, I'm sure you're captured by some possibility and magic of it. Do you read the work of others? Do you go through the proofs, or do you try to rediscover everything yourself?

So I think the things that I've.

Learned best and have the deepest ownership of are the ones that have some element of rediscovery.

The problem is that really slows you down.

And for my part, it's actually a big fault. This is part of why I'm not an active researcher. I'm not at the depth of the field. A lot of other people are. The stuff that I do learn, I try to learn it really well.

But other times you do need to.

Get through it at a certain pace. You do need to get to a point of a problem you're trying to solve. So obviously you need to be well equipped to read things without that reinvention component and see how others have done it.

But I think if you choose a few core building blocks along the way.

And you say, I'm really going to.

Try to approach this before I see.

How this person went at it, I'm really going to try to approach it for myself.

No matter what, you gain all sorts.

Of inarticulatable intuitions about that topic which.

Aren'T going to be there.

If you simply go through the proof.

For example, you're going to be trying to come up with counterexamples. You're going to try to come up with intuitive examples, all sorts of things.

Where you're Populating your brain with data.

And the ones that you come up.

With are likely to be different than the one that the text comes up with, and that lends at a different angle.

So that aspect also slowed Feynman down in a lot of respects.

I think there was a period when, like, the rest of physics was running away from him, but insofar as it got him to where he was, I kind of resonate with that.

I would be nowhere near it because.

I not like him at all. But it's like a state to aspire.

To just to link on a small point you made that you're not active.

Researcher.

You'Re swimming often in reasonably good depth about a lot of topics. Do you sometimes want to, like, dive deep at a certain moment and say, like. Because you probably built up a hell of an amazing intuition about what is and isn't true within these worlds. Do you ever want to just dive in and see if you can discover something new?

Yeah, I think one of my biggest.

Regrets from undergrad is not having built better relationships with the professors I had there. And I think a big part of success in research is that element of mentorship and people giving you the kind of scaffolded problems to carry along for my own goals. Right now I feel like I'm pretty good at exposing math to others and want to continue doing that for my personal learning. I.

Are you familiar with the hedgehog fox dynamic? I think this was either the ancient.

Greeks came up with it, or it was pretended to be something drawn from the ancient Greeks. I don't know who to point it.

To, but it's probably Mark Twain that.

Is that you've got two types of.

People or especially two types of researchers.

There's the fox that knows many different.

Things, and then the hedgehog that knows one thing very deeply. So Von Neumann would have been a fox. He's someone who knows many different things, just very foundational in a lot of different fields. Einstein would have been more of a hedgehog thinking really deeply about one particular.

Thing, and both are very necessary for making progress. So between those two, I would definitely.

See myself as the fox, where I'll try to get my paws in a whole bunch of different things.

And at the moment, I just think I don't know enough of anything to make a significant contribution to any of them. But I do see value in having a decently deep understanding of a wide variety of things.

Most people who know computer science really deeply don't necessarily know physics very deeply or many of the different fields in math.

Even let's say you have an analytic.

Number theory versus an algebraic number theory. These two things end up being related to very different fields. Some of them more complex analysis, some of them more like algebraic geometry.

And then when you just go out.

So far as to take those adjacent fields, place one PhD student into a seminar of another one, they don't understand what the other one's saying at all. You take the complex analysis specialist inside the algebraic geometry seminar, they're as lost as you or I would be. But I think going around and trying to have some sense of what this big picture is certainly has personal value for me. I don't know if I would ever make new contributions in those fields, but I do think I could make new expositional contributions where there's kind of a notion of things that are known but haven't been explained very well.

Well, first of all, I think most people would agree your videos, your teaching, the way you see the world is fundamentally often new. Like you're creating something new and it almost feels like research, even just like the visualizations, the multi dimensional visualization we'll talk about. I mean, you're revealing something very interesting that, yeah, just feels like research, feels like science, feels like the cutting edge of the very thing of which new ideas and new discoveries are made of.

I do think you're being a little.

Bit more generous than is necessarily, and.

I promise that's not even false humility, because I sometimes think when I research a video, I'll learn 10 times as much as I need for the video itself, and it ends up feeling kind of elementary. So I have a sense of just how far away the stuff that I.

Cover is from the actual depth.

I think that's natural, but I think that could also be a mathematics thing. I feel like in the machine learning world, two weeks in, you feel like you've basically mastered in mathematics.

It's like, well, everything is either trivial or impossible. And it's like a shockingly thin line between the two where you can find something that's totally impenetrable. And then after you get a feel for it, it's like, oh, yeah, that.

Whole subject is actually trivial in some.

Way, so maybe that's what goes on. And every researcher is just on the.

Other end of that hump and it feels like it's so far away, but one step actually gets them there.

What do you think about Feynman's teaching style or another perspective of use of visualization?

Well, his teaching style is interesting because people have described the Feynman effect Where.

While you're watching his lectures or while.

You'Re reading his lectures, everything makes such perfect sense as an entertainment session. It's wonderful because it gives you this.

Intellectual satisfaction that you don't get from anywhere else that you finally understand it.

But the Feynman effect is that you can't really recall what it is that gave you that insight even a week later. This is true of a lot of.

Books and a lot of lectures where the retention is never quite what we hope it is.

So there is a risk that the stuff that I do also fits that same bill where at best it's giving.

This kind of intellectual candy on giving a glimpse of feeling like you understand something.

But unless you do something active, like.

Reinventing it yourself, like doing problems to solidify it, even things like space, repetition, memory, to just make sure that you have the building blocks of what do all the terms mean?

Unless you're doing something like that, it's.

Not actually going to stick.

So the very same thing that's so.

Admirable about Feynman's lectures, which is how.

Damn satisfying they are to consume, might.

Actually also be reveal a little bit of the flaw that we should as educators all look out for, which is that that does not correlate with long term learning.

We'll talk about it a little bit. I think you've done some interactive stuff. I mean, even in your videos. The awesome thing that Feynman couldn't do at the time is you could, since it's programmed, you can tinker, play with stuff. You could take this value and change it. You can, like here, let's take the value of this variable and change it to build up an intuition to move along the surface or to change the shape of something. I think that's almost an equivalent of you doing it yourself. It's not quite there, but as a viewer. Yeah. Do you think there's some value in that interactive element?

Yeah. Well, so what's interesting is you're saying.

That and the videos are non interactive in the sense that there's a play button and a pause button and you.

Could ask, hey, while you're programming these things, why don't you program it into an interactable version, make it a jupyter.

Notebook that people can play with, which I should do. And that would be better.

I think the thing about interactives though is most people consuming them just sort.

Of consume what the author had in mind and that's kind of what they want.

I have a ton of friends who.

Make interactive explanations and when you look into the analytics of how people use them.

There's a small sliver that genuinely use.

It as a playground to have experiments.

And maybe that small sliver is actually.

Who you're targeting and the rest don't matter. But most people consume it just as a piece of well constructed literature that.

Maybe you tweak with the example a.

Little bit to see what it's getting at.

But in that way, I do think a video can get most of the.

Benefits of the interactive app, as long as you make the interactive for yourself and you decide what the best narrative to spin is. As a more concrete example, I made this video about SIR Models for Epidemics. And it's like this Agent Mace bottling thing where you tweak some things about how the epidemic spreads and you want to see how that affects its evolution. My format for making that was very different than others where rather than scripting.

It ahead of time, I just made the playground and then I played a bunch and then I saw what stories.

There were to tell within that.

Yeah, that's cool. So your video had that kind of structure. It had like five or six stories or whatever it was. And like it was basically, okay, here's a simulation, here's a model. What can we discover with this model? And here's five things I found after playing with it.

Well, because the thing is a way that you could do that project is you make the model and then you put it out and you say, here's a thing for the world to play with. Like, come to my website where you.

Interact with this thing and people did sort of remake it in a JavaScript way so that you can go to that website and you can test your own hypotheses.

But I think a meaningful part of the value to add is not just.

The technology, but to give the story around it as well.

That's kind of my job. It's not just to make the visuals.

That someone will look at.

It's to be the one to decide.

What'S the interesting thing to walk through here.

Even though there's lots of other interesting.

Paths that one could take that can.

Be kind of daunting when you're just.

Sitting there in a sandbox and you're given this tool with five different sliders and you're told to play and discover things.

Where do you do? What do you start? What are my hypotheses? What should I be asking? A little bit of guidance in that.

Direction can be what actually sparks curiosity to make someone want to imagine more about it.

A few videos I've seen you do.

I don't know how often you do it, but there's almost a tangential pause where you. Here's a cool thing. You say, here's a cool thing, but it's outside the scope of this video, essentially. But I'll leave it to you as homework, essentially to figure out. It's a cool thing to explore.

I wish I could say that wasn't.

A function of laziness.

And that's like, you've worked so hard.

On making the 20 minutes already that to extend it out even further would take more time.

One of your cooler videos, the homomorphic, like from the Mobius strip to the.

Or the described rectangle.

Yeah, that's a super. And you're like, yeah, you can't transform the Mobius strip into a surface without it intersecting itself. But I'll leave it to you to see why that is.

I hope that's not exactly how I phrase it, because I think what my hope would be is that I leave it to you to think about why you would expect that to be true and then to want to know, what aspects of a Mobius strip do you.

Want to formalize such that you can prove that intuition that you have?

Because at some point now you're starting to invent algebraic topology if you have.

These vague instincts like, I want to.

Get this Mobius strip, I want to fit it such that it's all above.

The plane, but its boundary sits exactly on the plane. I don't think I can do that without crossing itself.

But that feels really vague.

How do I formalize it? And as you're starting to formalize that, the. That's what's going to get you to.

Try to come up with a definition.

For what it means to be orientable or non orientable.

And once you have that motivation, a lot of the otherwise arbitrary things that.

Are sitting at the very beginning of a topology textbook start to make a little more sense.

Yeah. And I mean that whole video beautifully was a motivation for topology school.

Well, my hope with that is I feel like topology is. I don't want to say it's taught wrong, but I do think sometimes it's popularized in the wrong way where you'll hear these things of people saying, oh, topologists, they're very interested in surfaces that you can bend and stretch, but you can't cut or glue. Are they? Why there's all sorts of things you can be interested in with random, imaginative manipulations of things. Is that really what mathematicians are into? And the short answer is not really, it's not as if someone was sitting there thinking, I wonder what the properties of clay are if I had some arbitrary rules about when I can't cut.

It and when I can't glue it.

Instead, there's a ton of pieces of math that can actually be equivalent to.

These very general structures. That's like geometry, except you don't have exact distances. You just want to maintain a notion of closeness.

And once you get it to those general structures, constructing mappings between them translate.

Into non trivial facts about other parts of mathematics.

And that, I don't think that's actually popularized. I don't even think it's emphasized well.

Enough when you're starting to take a.

Topology class because you kind of have these two problems. It's like either it's too squishy, you're.

Just talking about coffee mugs and donuts.

Or it's a little bit too rigor first.

And you're talking about the axiom systems with open sets. And an open set is not the.

Opposite of closed set.

So sorry about that, everyone. We have a notion of Clopin sets.

For ones that are both at the same time. And it's not an intuitive axiom system in comparison to other fields of math. So you as the student really have.

To walk through mud to get there.

And you're constantly confused about how this relates to the beautiful things about coffee.

Mugs and Mobius strips and such.

And it takes a really long time to actually see topology in the way.

That mathematicians see topology.

But I don't think it needs to take that time.

I think there's.

This is making me feel like I.

Need to make more videos on the.

Topic because I think I've only heard 100% you do.

But I've also seen it in my narrow view of. I find game theory very beautiful and I know topology has been used elegantly to prove things. In game theory.

Yeah. You have facts that seem very strange. Like I could tell you, you stir.

Your coffee and after you stir it, and let's say all the molecules settle to not moving again, one of the molecules will be basically in the same position it was before.

You have all sorts of fixed point.

Theorems like this, that kind of fixed.

Point theorem directly relevant to Nash Equilibriums. So you can imagine popularizing it by describing the coffee fact, but then you're left to wonder, who cares about if a molecule of coffee stays in the same spot? Is this what we're paying our mathematicians for? You have this very elegant mapping onto economics in a way that's very concrete. I shouldn't say concrete.

Very tangible.

Actually adds value to people's lives through.

The predictions that it makes.

But that line isn't always drawn because you have to get a little bit.

Technical in order to properly draw that line out. And often, I think popularized forms of media just shy away from being a little too technical. For sure.

By the way, for people who are watching the video, I do not condone the message on this mug. It's the only one I have, which is the snuggle is real.

By the way, for anyone watching, I do condone the message of that mug.

The snuggle is real.

The snuggle is real.

Okay, so you mentioned the SIR model. I think there are certain ideas there of growth, of exponential growth. What maybe have you learned about pandemics from making that video? Because it was kind of exploratory. You were kind of building up an intuition. And it's again, people should watch the video. It's kind of an abstract view. It's not really modeling in detail the whole field of epidemiology. Those, those people, they go really far in terms of modeling, like how people move about. I don't know if you've seen it, but like there is the mobility patterns. Like how, like how many people you encounter in certain situations when you go to a school, when you go to a mall, they like model every aspect of that for a particular city. Like they have maps of actual city streets. They model it really well. And natural patterns of the people have. Is crazy. So you don't do any of that. You're just doing an abstract model to explore different ideas of simple pedagogy.

Well, because I don't want to pretend.

Like I'm an epidemiologist. We have a ton of armchair epidemiologists.

And the spirit of that was more.

Like, can we, through a little bit of play, draw reasonable ish conclusions and.

Also just get ourselves in a position where we can judge the validity of a model?

I think people should look at that and they should criticize it. They should point to all the ways.

That it's wrong because it's definitely naive.

In the way that it's set up. But to say what lessons from that hold, Thinking about the R naught value and what that represents and what it can imply.

What's R naught?

So R naught is if you are.

Infectious and you're in a population which is completely susceptible, what's the average number of people that you're going to infect during your infectiousness? So Certainly during the beginning of an epidemic. This basically gives you kind of the exponential growth rate. If every person infects two others, you've got that 1, 2, 4, 8 exponential growth pattern as it goes on. And let's say it's something endemic where you've got a ton of people who have had it and are recovered. Then the R0 value doesn't tell you that as directly because a lot of the people you interact with aren't susceptible. But in the early phases it does.

And this is the fundamental constant that.

It seems like epidemiologists look at. And the whole goal is to get that down. If you can get it below one, then it's no longer epidemic. If it's equal to one, then it's endemic and it's above one, then you're epidemic. So just teaching what that value is and giving some intuitions on how do certain changes in behavior change that value? And then what does that imply for exponential growth? I think those are general enough lessons and they're resilient to all of the chaos of the world. It's still valid to take from the video.

I mean, one of the interesting aspects of that is just exponential growth and we think about growth. Is that one of the first times you've done a video on. No, of course not. The whole Euler's identity. Okay.

So sure, I guess I've done a lot of videos about exponential growth in the circular direction, only minimal in the normal direction.

I mean, another way to ask, do you think we're able to reason intuitively about exponential growth?

It's funny, I think it's extremely intuitive to humans. And then we train it out of.

Ourselves such that it's then really not intuitive.

And then I think it can become.

Intuitive again when you study a technical field.

So what I mean by that is, have you ever heard of these studies.

Where in an anthropological setting where you're studying a group that has been disassociated from a lot of modern society and you ask what number is between one and nine? And maybe you would ask, you've got one rock and you've got nine rocks. You're like, what pile is halfway in between these? And our instinct is usually to say five. That's the number that sits right between one and nine. But sometimes when numeracy and the kind of just basic arithmetic that we have isn't.

In a society, the natural instinct is.

Three because it's in between in an exponential sense and a geometric sense, that one is three times bigger. And then the next one is three times bigger than that. So it's like if you have one friend versus 100 friends, what's in between that? Yeah, 10 friends seems like the social status in between those two states.

So that's deeply intuitive to us.

Think logarithmically like that. And for some reason we kind of train it out of ourselves to start thinking linearly about things.

So in the sense, yeah, the early basic math forces us to take a step back. It's the same criticism if there's any of science is the lessons of science make us see the world in a slightly narrow sense to where we have an over exaggerated confidence that we understand everything as opposed to just understanding a small slice of it.

But I think that probably only really goes for small numbers because the real.

Counterintuitive thing about exponential growth is as the numbers start to get big.

So I bet if you took that.

Same setup and you asked them, oh, if I keep tripling the size of this rock pile seven times, how big will it be? I bet it would be surprisingly big, even to a society without numeracy.

And that's the side of it that I think is pretty counterintuitive to us.

But that you can basically train into people.

I think computer scientists and physicists, when.

They'Re looking at the early numbers of COVID they were the ones thinking like.

Oh God, this is following an exact exponential curve. And I heard that from a number of people. And almost all of them are techies.

In some capacity, probably just because I live in the Bay Area, but, but.

For sure they're cognizant of this kind of, this kind of growth is present in a lot of natural systems. In a lot of, in a lot of, in a lot of systems. I don't know if you've seen like, I mean, there's a lot of ways to visualize this, obviously, but Ray Kurzweil I think was the one that had this like, chessboard where every, every square in the chessboard, you double the number of stones or something in that chessboard.

I've heard this is like an old.

Proverb where, you know, someone, the king offered him a gift and he said, the only gift I would, very modest, give me a single grain of rice for the first chessboard and then two grains of rice for the next square and then twice that for the next square and just continue on. That's my only modest. Ask your sire. And it's all more grains of rice than there are anything in the world by the time you get to the end.

And my intuition Falls apart there. I would have never predicted that for some reason. That's a really compelling illustration. How poorly breaks down. Just like you said, maybe we're okay for the first few piles of rocks, but after a while it's game over.

You know the other classic example for.

Gauging someone's intuitive understanding of exponential growth is I've got like a lily pad on lake, really big lake. Okay, Like Lake Michigan. And that lily pad replicates. It doubles one day and then it doubles the next day, and it doubles the next day. And after 50 days, it actually is going to cover the entire lake. So after how many days does it cover half the lake?

49.

So you have a good instinct for exponential growth. So I think a lot of the knee jerk reaction is sometimes to think that it's half the amount of time or to at least be surprised that after 49 days you've only covered half of it.

Yeah, I mean, the reason you heard a pause for me, I literally thought that can't be.

Right, Right.

Yeah, exactly. So even when you know the fact and you do the division, it's like, wow, so you've gotten like that whole time, and then day 49, it's only covering half, and then after that it.

Gets the whole thing.

But I think you can make that even more visceral if rather than going.

One day before you say how long until it's covered 1% of the lake? Right. So what would that be? How many times you have to double to get over 100? Like seven, six and a half times, something like that.

So at that point, you're looking at.

43, 44 days into it. You're not even at 1% of the lake.

So you've experienced 44 out of 50 days. And you're like, yeah, that lily pad is just 1% of the lake.

But then next thing you know, it's the entire lake.

You're wearing a SpaceX shirt. So let me ask you.

Sure.

Let me ask you. One person who talks about exponential.

Just.

The miracle of the exponential function in general is Elon Musk. So he kind of advocates the idea of exponential thinking, realizing that technological development can, at least in the short term, follow exponential improvement, which breaks apart our intuition, our ability to reason about what isn't, isn't impossible. So he's a big one. It's a good leadership kind of style of saying, like, look, the thing that everyone thinks is impossible is actually possible because exponentials. But what's your sense about that kind of way to see the world?

Well, so I think it can be.

Very inspiring to note when something like Moore's Law is another great example where you have this exponential pattern that holds shockingly well and it enables just better lives to be led. I think the people who took Moore's Law seriously in the 60s were seeing.

That, wow, it's not going to be.

Too long before these giant computers that are either batch processing or timeshared, you.

Could actually have one small enough to.

Put on your desk, on top of.

Your desk, and you could do things. And if they took it seriously, you.

Have people predicting smartphones a long time ago. And it's only out of this.

I don't want to say faith in.

Exponentials, but an understanding that that's what's happening.

What's more interesting, I think, is to.

Really understand why exponential growth happens, and that the mechanism behind it is when the rate of change is proportional to the thing in and of itself. So the reason that technology would grow exponentially is only going to be if the rate of progress is proportional to the amount that you have, so that the software you write enables you to write more software.

I think we see this with the.

Internet, like the advent of the Internet makes it faster to learn things, which makes it faster to create new things. I think this is oftentimes why investment will grow exponentially, that the more resources a company has, if it knows how to use them well, the more it can actually grow. So, I mean, you reference Elon Musk.

I think he seems to really be.

Into vertically integrating his companies.

I think a big part of that is because you have the sense, what you want is to make sure that the things that you develop, you have ownership of, and they enable further development.

Of the adjacent parts.

So it's not just this. You see a curve and you're blindly.

Drawing a line through it. What's much more interesting is to ask, when do you have this proportional growth property?

Because then you can also recognize when.

It breaks down, like in an epidemic, as you approach saturation, that would break down. As you do anything that skews what that proportionality constant is, you can make it maybe not break down as being an exponential, but it can seriously slow what that exponential rate is.

The opposite of a pandemic is you want, in terms of ideas, you want to minimize barriers that prevent the spread. You want to maximize the spread of impact. So you want it to grow when you're doing technological development so that you do hold up, that rate holds up. And that's almost like. Like an operational challenge of, like, how you run a company, how you run a group of people is that any one invention has a ripple that's unstopped and that ripple effect then has its own ripple effects and so on, and that continues. Yeah, like Moore's Law is fascinating on a psychological level, on a human level, because it's not exponential. It's just a consistent set of what you would call S curves, which is like it's constant. Constantly breakthrough innovations nonstop.

That's a good point. It might not actually be an example of exponentials because of something which grows in proportion to itself, but instead it's almost like a benchmark that was set out that everyone's been pressured to meet. And it's like all these innovations and.

Micro inventions along the way, rather than some consistent sit back and just let the lily pad grow across the lake phenomenon.

And it's also, there's a human psychological level for sure of the four minute mile. It's, there's something about it like saying that look, there is, you know, Moore's law, it's a law. So like it's, it's certainly an achievable thing. You know, we've achieved it for the last decade, for the last two decades, for last three decades. You just keep going and it somehow makes it happen. I mean it makes people. I'm continuously surprised in this world how few people do the best work in the world, like in that particular, whatever that field is. Like it's very often that like the genius, I mean you couldn't argue that community matters, but it's certain. Like I've been in groups of engineers where like one person is clearly like doing an incredible amount of work and just is the genius. And it's fascinating to see. Basically it's kind of the Steve Jobs idea is maybe the whole point is to create an atmosphere where the genius can discover themselves, like have the opportunity to do the best work of their life. And the exponential is just milking that. It's like rippling the idea that it's possible. And that idea that it's possible finds the right people for the 4 minute mile. The idea that it's possible finds the right runners to run it, and then explodes the number of people who can run faster than four minutes. It's kind of interesting to. I don't know, basically the positive way to see that is most of us are way more intelligent, have way more potential than we ever realize. I guess that's kind of depressing. But I mean like the ceiling for most of us is much higher than we ever realized.

That is true. A good book to read if you want that sense is Peak, which essentially talks about peak performance in a lot of different ways, like chess, London, cab drivers, how many push ups people can do short term memory tasks. It's meant to be a concrete manifesto about deliberate practice and such, but the.

One sensation you come out with is wow. No matter how good people are at something, they can get better and way.

Better than we think they could.

I don't know if that's actually related to exponential growth, but I do think it's a true phenomenon.

That's interesting.

Yeah, I mean there's certainly no law of exponential growth in human innovation. Well, I don't know.

Well, kind of there is. I think it's really interesting to see when innovations in one field allow for innovations in another. Like the advent of computing seems like.

A prerequisite for the advent of chaos theory.

You have this truth about physics and the world that in theory could be known.

You could find Lorenz's equations without computers.

But in practice it was just never going to be analyzed that way unless.

You were doing a bunch of simulations and that you could computationally see these models. So it's like physics allowed for computers.

Computers allowed for better physics.

And wash, rinse and repeat. That self proportionality, that's exponential. So I think.

I wouldn't think it's too far to say that that's a.

Law of some kind. Yeah.

A fundamental law of the universe is that these descendants of apes will exponentially improve their technology and one day be taken over by the AGI that's built in the simulator that'll make the video game fun. Whoever created this thing. Since you're wearing a SpaceX shirt, let me ask.

I didn't realize that. I apologize. It's on topic.

Yeah.

So, crew Dragon, the first crewed mission out into space since the space shuttle. And just by first time ever by a commercial company. I mean it's an incredible accomplishment, I think, but it's also just an incredible. It inspires imagination amongst people that this is the first step in a long vibrant journey of humans into space.

Oh yeah.

So how do you feel? Is this exciting to you?

Yeah, it is.

I think it's great. The idea of seeing it basically done.

By smaller entities instead of by governments.

I mean it's a heavy collaboration between.

SpaceX and NASA in this case, but moving in the direction of not necessarily requiring an entire country and its government to make it happen, but that you can have something closer to a single company doing it. We're not there yet, because it's not like they're unilaterally saying we're just shooting people up into space.

It's just a sign that we're able to do more powerful things with smaller groups of people.

I find that inspiring.

Innovate quickly.

I hope we see people land on Mars in my lifetime.

Do you think we will?

I think so. I think there's a ton of challenges.

There, like radiation being kind of the biggest one.

And I think there's a ton of people who look at that and say why?

Why would you want to do that? Let's let the robots do the science for us.

But I think there's enough people who are genuinely inspired about broadening the worlds that we've touched.

Or people who think about things like backing up the light of consciousness with super long term versions of terraforming.

As long as they're backing up the light of consciousness.

Yeah.

The thought that if Earth goes to hell, we got to have a backup somewhere.

A lot of people see that as.

Pretty out there and it's not in the short term future, but I think.

That'S an inspiring thought. I think that's a reason to get.

Up in the morning. And I feel like Most employees at SpaceX feel that way too.

Do you think we'll colonize Mars one day?

No idea. Either AGI kills us first or if we're allowed.

I don't know if it'll take us if we're allowed. Well, honestly, it would take such a long time. Okay.

You might have a small colony, something like what you see in the Martian, but not people living comfortably there.

But if you want to talk about.

Actual second Earth kind of stuff, that's just way far out there.

And the future moves so fast that.

It'S hard to predict.

We might just kill ourselves before that even becomes viable.

Yeah, I mean there's a lot of possibilities where it could be just. It doesn't have to be on a planet. We could be floating out in space. Have, have, have a, have a space faring backup solution that doesn't have, that doesn't have to deal with the constraints that a planet. I mean a planet provides a lot of possibilities and resources, but also has some constraints. Yeah. I mean, for me, for some reason it's a deeply exciting possibility.

Oh yeah.

All of the people who are like skeptical about it are like, why? Why do we care about going to Mars? It's like what makes you care about.

Anything if that's inspiring?

It's hard actually.

It's hard to hear that because Exactly. As you put it on a philosophical level, it's hard to say why do anything. I don't know, it's. It's like the people say like, you know, I've been doing like an insane challenge last 30 something days.

Your pull ups and the pull ups.

And push ups and like, you know, a bunch of people are like awesome. You're insane but awesome. And then some people are like, why?

Why do anything?

I don't know. There's a calling, it's, I'm with JFK a little bit is because we do these things because they're hard. There's something in the human spirit that says like same with like a math problem. There's something, you fail once and it's like this feeling that, you know what, I'm not going to back down from this. There's something to be discovered in overcoming this thing.

Well, so what I like about it is, and I also like this about the moon missions. Sure it's kind of arbitrary, but you can't move the target. So you can't make it easier and.

Say that you've accomplished the goal.

And when that happens it just demands actual innovation.

Right.

Like protecting humans from the radiation in.

Space on the flight there.

While there hard problem demands innovation, you can't move the goalpost to make that easier. Almost certainly the innovations required for things like that will be relevant in a.

Bunch of other domains too.

So the idea of doing something merely because it's hard, it's loosely productive, great. But as long as you can't move the goalposts, there's probably going to be.

These secondary benefits that we should all strive for.

Yeah, I mean it's hard to formulate the Mars colonization problem as something that has a deadline, which is the problem. But if there was a deadline, then the amount of things we would come up with by forcing ourselves to figure out how to colonize that place would be just incredible. This is what people like the Internet didn't get created because people sat down and tried to figure out how do I send TikTok videos of myself dancing to people they, you know, it was, there's an application. I mean, actually, I don't even know.

What do you think the application for the Internet was when it was, it.

Must have been very low level basic network communication within darpa, like military based. Like how do I send like a networking, how do I send information securely between two places? Maybe it was an encryption. I'm totally speaking totally outside of my knowledge, but like it was probably intended for a very narrow Small group of people.

Well, so, I mean, there was like this small community of people who were.

Really interested in timesharing computing and interactive computing, in contrast with batch processing.

And then the idea that as you.

Set up a timesharing center, basically meaning you can have multiple people logged in and using that central computer, why not make it accessible to others?

And this was kind of what I had always thought like, oh, is this fringe group that was interested in this.

New kind of computing and they all got themselves together.

But the thing is, darpa, you wouldn't have the US Government funding that just.

For the funds of it.

In some sense, that's what ARPA was.

All about, was just really advanced research for the sake of having advanced research. And it doesn't have to pay out with utility soon.

But the core parts of its development were happening in the middle of the.

Vietnam War when there was budgetary constraints all over the place.

I only learned this recently, actually. If you look at the documents basically justifying the budget for the arpanet as they were developing it, and not just keeping it where it was, but actively.

Growing it while all sorts of other departments were having their funding cut because.

Of the war, a big part of it was national defense in terms of.

Having a more robust communication system, like the idea of packet switching versus circuit switching. You could kind of make this case that in some calamitous circumstance where a central location gets nuked, this is a much more resilient way to still have your communication lines that traditional telephone lines weren't as resilient to, which I just found very interesting.

Even something that we see as so.

Happy go lucky is just a bunch of computer nerds trying to get interactive computing out there.

The actual thing that made it funded.

And thing that made it advance when it did was because of this direct national security question and concern.

I don't know if you've read it. I haven't read it. I've been meaning to read it, but Neil Degrasse Tyson actually came out with a book that talks about science in the context of the military, basically saying all the great science we've done in the 20th century was because of the military. I mean, he paints a positive. It's not like a critical. A lot of people say military industrial complex and so on. Another way to see the military and national security is a source of, like you said, deadlines and hard things you can't move. Almost like scaring ourselves into being productive.

It is that. I mean, Manhattan Project is a perfect.

Example, probably the quintessential Example, that one is a little bit more macabre than others because of what they were building.

But in terms of how many focused.

Smart hours of human intelligence get pointed towards a topic per day, you're just maxing it out with that sense of worry. In that context, everyone there was saying, we've got to get the bomb before.

Hitler does, and that just lights a fire under you. That, again, the circumstances, macabre. But I think that's actually pretty healthy, especially for researchers that are otherwise going.

To be really theoretical. To take these theorizers and say, make.

This real physical thing happen, meaning a lot of it is going to be unsexy. A lot of it's going to be young Feynman sitting there kind of inventing a notion of computation in order to.

Compute what they needed to compute more quickly with the rudimentary automated tools that they had available.

I think you see this with Bell.

Labs also, where you've got otherwise very theorizing minds in very pragmatic contexts that I think is really helpful for the theory as well as for the applications. So I think that stuff can be positive for progress.

You mentioned Bell Labs and Manhattan Project. This kind of makes me curious for the things you create, which are quite singular. Like, if you look at all YouTube or just not YouTube, it doesn't matter what it is. It's just teaching content. Art doesn't matter. It's like, yep, that's. That's Grant. Right? That's unique and your teaching style and everything. Does it? Manhattan Project and Bell Labs was like, famously, a lot of brilliant people, but there's a lot of them. They play off of each other. So, like, my question for you is, does it get lonely?

Honestly, that right there, I think, is the biggest part of my life that I would like to change in some.

Way that I look at a Bell Labs type situation, and I'm like, God.

Damn, I love that whole situation, and.

I'm so jealous of it.

And you're reading about hamming, and then you see that he also shared an office with Shannon, and you're like, of course he did. Of course they shared an office. That's how these ideas get.

And they actually very likely worked separately.

Yeah, totally, totally separate.

But there's a literally. And sorry to interrupt. There's literally magic that happens when you run into each other on the way to getting a snack or something, conversations you overhear.

It's other projects you're pulled into. It's like puzzles that colleagues are sharing.

All of that, I have some extent of it just Because I try to stay well connected in communities of people who think in similar ways, but it's not in the day to day in the same way, which I would like to fix somehow.

That's one of the, I would say one of the biggest. Well, one of the many drawbacks, negative things about this current pandemic is that whatever the term is, but like chance collisions are significantly reduced.

I don't know why I saw this.

But on my brother's work calendar he.

Had a scheduled slot with someone that he scheduled a meeting. And the title of the whole meeting was no specific agenda. I just miss the happensance, serendipitous conversations that we used to have, which the.

Pandemic and remote work has so cruelly.

Taken away from us.

Brilliant.

That was the whole title of the meeting.

That's brilliant.

I'm like, that's the way to do it. You just schedule those things. Schedule the serendipitous interaction.

It's like, I mean you can't do it in an academic setting, but it's basically like going to a bar and sitting there just for the strangers you might meet just as strangers or striking up conversations with strangers on the train. Harder to do when you're deeply like maybe myself or maybe a lot of academic types who are like introverted and avoid human contact as much as possible. So it's nice when it's forced those chance collisions, but maybe scheduling is a possibility. But for the most part. Do you work alone? Like I'm sure you struggle like a lot like, like this, like you probably hit moments when you it you look at this and you say like this is the wrong way to show it, it's the wrong way to visualize it. I'm making it too hard for myself. I'm going down the wrong direction. This is too long, this is too short. All those self doubt that's like could be paralyzing. Like what do you do in those moments?

Honestly, I actually much prefer work to.

Be a solitary affair. For me that's like a personality quirk. I would like it to be in an environment with others and collaborative in the sense of ideas exchanged.

But those phenomena you're describing when you.

Say this is too long, this is too short, this visualization sucks.

It's way easier to say that to.

Yourself than it is to say to a collaborator. And I know that's just a thing that I'm not good at. So in that way it's very easy to just throw away a script because the script isn't working. It's Hard to tell someone else they should do the same.

Actually, last time we talked, I think it was, like, very close to me talking. Don Knuth was kind of cool. Like, two people that I believe.

You got that interview. It's the heart.

No. Can I brag about something?

Please.

My favorite thing is Don Knuth, after.

Did the interview, he offered to go.

Out to hot dogs with me to get hot dogs. That was never, like. People ask me, what's the favorite interview you've ever done? Man, that has to be. But unfortunately, I couldn't. I had a thing after, so I had to turn down Don Knuth.

You missed Knuth Dogs.

Knuth dogs. Sorry. So that was a little bragging. But the hot dogs, he says, just sweet. So. But the reason I bring that up is he works through problems alone as well. He prefers that struggle. The struggle of it. You know, writers like Stephen King, you know, often talk about, like, their process of, you know, what they do, like what they eat when they wake up, like when they sit down, like how they like their desk, you know, on a. On a perfectly productive day. Like, what they like to do, how long they like to work for, what enables them to think deeply, all that kind of stuff. Hunter Thompson did a lot of drugs. You know, everybody has their own thing. Do you have a thing? If you were to lay out a perfect productive day, what would that schedule look like, do you think?

Part of that's hard to answer because the mode of work I do changes a lot from day to day. Some days I'm writing.

The thing I have to do is write a script. Some days I'm animating. The thing I have to do is animate. Sometimes I'm working on the animation library. The thing I have to do is a little. I'm not a software engineer, but something in the direction of software engineering. Some days it's like a variant of research. It's like, learn this topic well and try to learn it differently.

So those are four very different modes of what it some days is like get through the email backlog of people I've been. Tasks I've been putting off.

It goes research, scripting. The idea starts with research, and then there's scripting, and then there's programming, and then there's the showtime and the research.

Side, by the way, I think a.

Problematic way to do it is to say, I'm starting this project and therefore I'm starting the research.

Instead, it should be that you're ambiently.

Learning a ton of things just in the background. And then once you Feel like you have the understanding for one, you put it on the list of things that there can be a video for.

Otherwise, either you're going to end up.

Roadblocked forever or you're just not going to have a good way of talking about it.

But still, some of the days it's.

Like the thing to do is learn new things.

So what's the most painful one? I think you mentioned scripting.

Scripting is. Yeah, that's the worst. Yeah, writing is the worst.

So what's your. On a perfectly.

So let's take the hardest one.

What's a perfectly productive day?

You wake up and it's like, damn it, this is the day I need to do some scripting. And like, you didn't do anything last two days, so you came up with excuses to procrastinate. So today must be the day.

Yeah, I wake up early, I guess I exercise and then I turn the Internet off.

If we're writing.

Yeah, that's what's required is having the Internet off.

And then maybe you keep notes on.

The things that you want to Google when you're allowed to have the Internet.

Again, I'm not great about doing that.

But when I do, that makes it happen.

And then when I hit writer's block. The solution to writer's block is to read. It doesn't even have to be related.

Just read something different. Just for like 15 minutes, half an hour, and then go back to writing that when it's a nice cycle, I think can work very well.

And when you're writing the script, you don't know where it ends.

Right.

Problem solving videos.

I know where it ends. Expositional videos. I don't know where it ends.

Coming up with the magical thing that makes this whole story, ties this whole story together. When does that happen?

That's the thing that makes it such.

That a topic gets put on the list of videos.

Oh, that's an issue.

You shouldn't start the project unless there's one of those.

And you have so many nice bags. You have such a big bag of aha moments already that you could just pull at it. That's one of the things and one of the sad things about time and that nothing lasts forever and that we're all mortal. Let's not get into that discussion. Is, you know, if I see, like, even when I ask for people to ask, like, ask. I did a call for questions and people want to ask you questions. And so many requests from people about, like, certain videos they would love you to do.

It's such a Pile.

And I think that's a, that's a sign of like admiration from people for sure. But it's like it makes me sad because like whenever I see them, people give ideas. They're all like very often really good ideas. And it's like, it's such a. Makes me sad in the same kind of way. When I go through a library or through a bookstore. You see all these amazing books that you'll never get to open. So yeah, enjoy the ones that you have.

Enjoy the books that are open and don't let yourself lament the ones that stay closed.

What else?

Is there any other magic to that day? Do you try to dedicate a certain number of hours? Cal Newport has this deep work kind of idea.

There's systematic people who get really on.

Top of the checklist of what they're going to do in the day and they count their hours. And I am not a systematic person.

In that way, which is probably a problem. I very likely would get more done.

If I was systematic in that way. But that doesn't happen. So you talk to me later in.

Life and maybe I will have changed.

My ways and give you a very different answer.

I think Benjamin Franklin later in life figured out the rigor. He has these very rigorous schedules and how to be productive.

I think those schedules are much more fun to write. It's very fun to write a schedule and make a blog post about the.

Perfect productive day that might work for one person. But I don't know how much people get out of reading them or trying to adopt someone else's style.

And I'm not even sure that they've ever followed.

Yeah, exactly. You're always going to write it as.

The best version of yourself.

You're not going to explain the phenomenon of wanting to get out of the.

Bed but not really wanting to get out of the bed and all of.

That and just zoning out for random reasons. Or the one that people probably don't touch at all is I try to check social media once a day, but only so I post and that's it. When I post, I check the previous days. That's like my what I try to do. That's why I do like 90% of the days. But then I'll go, I'll have like a two week period where it's just like I'm checking the Internet. Like, I mean it's some, probably some scary number of times.

I think a lot of people can resonate with that. I think it's a legitimate addiction. It's like it's a dopamine addiction.

And I don't know if it's a problem because as long as it's a.

Kind of socializing, like, if you're actually engaging with friends and engaging with other people's ideas, I think it can be really useful.

Well, I don't know. So for sure, I agree with you. But it's definitely an addiction because for me, I think it's true for a lot of people. I am very cognizant of the fact I just don't feel that happy. If I look at a day where I've checked social media a lot, like, if I just aggregate, I did a self report, I'm sure I would find that I'm just, like, literally on, like, less happy with my life and myself after I've done that check. When I check it once a day, I'm very, like, I'm happy even, like, because I've seen it. Okay. One way to measure that is when somebody says something not nice to you on the Internet is like, when I check it once a day, I'm able to just, like, I smile. I virtually. I think about them positively, empathetically. I send them love. I don't, don't ever respond, but I just feel positively about the whole thing. If I check, like, more than that, it starts eating at me. Like, there's an eating thing that happens, like anxiety. It occupies a part of your mind that's not. Doesn't seem to be healthy. Same with. I mean, you, you. You put stuff out on YouTube. I think it's important. I think you have a million dimensions that are interesting to you. But yeah, one. One of the interesting ones is the study of education and the psychological aspect of putting stuff up on YouTube. I, like, now have completely stopped checking statistics of any kind. I've released an episode 100 with my dad conversation with my dad. He checks. He's probably listening to this. Stop. He checks the number of views on his. On his video, on his conversation. So he discovered, like, a reason he's new to this whole addiction. And he just checks and he, like, he'll text me or write to me. I just passed Dawkins in the top.

Oh, my God, I love that so much.

Yeah.

So he's.

Can I tell you a funny story in that effect of parental use of YouTube? Early on in the channel, my mom would text me. She's like, the channel has had 990,000 views. The channel has had 991,000 views. I'm like, oh, that's cute.

She's going to the little part on.

The about page where you see the.

Total number of channel views.

No, she didn't know about that. She had been going every day through all the videos and then adding them.

Up, adding them up.

And she thought she was doing me this favor of providing me this global analytic that otherwise it wouldn't be visible. That's awesome. It's just like this addiction where you have some number you want to follow and then, yeah, it's funny that your dad had this.

I think a lot of people have it.

I think that's probably a beautiful thing for parents because they're legitimately. They're proud.

It's born of love.

It's great.

The downside, I feel one of them is this is one interesting experience that you probably don't know much about because comments on your videos are super positive. But people judge the quality of how something went. Like, I see that with these conversations by the comments.

Yeah, like.

And I'm not talking about, like, you know, people in their 20s and their 30s. I'm talking about, like, CEOs of major companies who don't have time. They basically, they literally. This is their evaluation metric. They're like, ooh, the comments seem to be positive. And that's really concerning to me.

Most important lesson for any content creator to learn is that the commenting public is not representative of the actual public.

And this is easy to see.

Ask yourself, how often do you write comments on YouTube videos?

Most people will realize I never do it. Some people realize they do, but the people who realize they never do it should understand that that's a sign. The kind of people who are like you aren't the ones leaving comments. And I think this is important in a number of respects. In my case, I think I would think my content was better than it.

Was if I just read comments because.