Scientist Stories: Jeff Litchman, Connectomics Across Scales

I would like to welcome Jeff Lichtman to the Brains, Minds, and Machines summer school. Jeff is a professor of molecular and cellular biology at Harvard. Jeff is a remarkable scientist whose research I think has the potential to transform our understanding of biology, and we all are very much looking forward to hear Jeff's lecture. Thank you for being here. Okay. I'm going to be giving a lecture that is going to be both looking backwards in time and perhaps forward in time. I'm assuming that not every one of you is a card-carrying neurobiologist. Is that fair to say? So if I do say things that sound like you don't understand them, you should definitely stop me and say, "I don't understand what you're saying." There's no point in me blabbing on if you don't understand what I'm saying. You don't actually have to be much of a biologist to know some things about biology, even if some of you aren't. And that one of the central tenets of biology of animals like us is that we have a bunch of organs in our body called vital organs, and they're called vital because you can't live without them. And among them are your kidneys, your heart, obviously, your liver, pancreas, and one that is very special to me. I was gonna say close to my heart, but it's actually close to my head. And that is the brain. And I think although we all appreciate that every one of these vital organs is vital, the one that is most special is most special for a very good reason. And I'll give you a very straightforward way of appreciating how important it is. If you went to your doctor and they did some tests of your kidney function and discovered that your kidneys are failing, The doctor would say, I've got bad news for you, your kidneys are failing, but I have good news for you, I can give you the kidney of a cadaver, a recently deceased person's kidney. I can give you that kidney and you will then be able to live for a long time. And I think most of you will probably take up the doctor's offer for a new kidney. If it meant your life. Similarly, if you had a very serious problem with your liver, now liver transplants are possible, and I think most of us would happily take someone else's liver to live. And even someone's heart or lungs, both of those things, they're a little more adventurous to get heart transplants and lung transplants, but they're done. But if your doctor said to you, you know, your body is doing pretty well, but your brain "Hey, it's just not working the way it is. You have a disease, your brain is not gonna work, but there's good news. I can put someone else's brain in there and then you'll be like new." I think most of us would probably say, "Nah, I'll just stick with my own brain." Now why is that? Why is the brain an organ that we feel more attached to than these other organs. And I think it's because of all the organs we have, it's the only one that has personhood. It's part of us in a way that, you know, one person's liver versus your own liver, you don't really know the difference. But your brain is special for a very important reason, and that is it has recorded all the history of your life. You put someone else's brain into this a skull, it's not going to be you. It may look like you, but it isn't you. The brain is what holds this information. And this raises a really interesting question. Why does this organ have personhood? How does it manage to have a record of who we are that, if you are lucky, may even last 100 years? That is, you will still have installed in that brain information that was acquired when you were a very young child and it's still there. And that is a deeply interesting question because we know a lot about memory systems. We know memory in computers. It's a very stable form of memory and it can sit for a long time quietly and then be brought back to life in your computer. We know about memory in DNA. You come into this world most of the time with 5 fingers because somehow your DNA encodes for a body plan that has 5 fingers. That information, which is based on the memory of the ages, eons of memory, gives you these physical features. But this other kind of memory, the memory that we hold in our heads, is a memory we do not yet understand. But it's not any more magical than these other forms. It almost certainly is a physical form of memory, but what is it? When you know, let's say, what the American flag looks like, where is that in your brain? How much does the American flag memory weigh? What is it physically? What are we talking about? Nobody knows. It's ridiculous how little we really know about this. There's all sorts of theories. But nobody really knows. And this is sort of the central question of my own interest. Is everybody sort of clear on this? Okay. I want to start by emphasizing something that's exactly the opposite of what most neuroscientists talk about. Most neuroscientists who talk about learning are really interested in synaptic plasticity, the way the brain changes. But if you think about it, memories, are signs of indelible changes to the brain that don't change. You don't— once you learn that language, it's learned. It's indelible. It's not going to change. It's the opposite of plasticity. It's kind of stable and very difficult to get rid of. You can superimpose new memories. You can reconsolidate memories. But to First approximation, memories that haven't been altered just sit there indelible. They're not changed by other experiences. So I'm going to start by giving you a couple of examples of these kinds of evidence of this fact that a lot of what we learn is kind of in there in a way that is unchangeable. And I'm going to start with a kind of motor memory, and then I'm going to talk about a sensory memory, and then finally I'm going to talk about a multimodal memory. And I'm going to take for my data here the greatest compendium of information humans have ever assembled, and that is of course YouTube. So we're going to look at 3 YouTube videos. I'm going to start with these, and I'll let them narrate them. Themselves. We'll start with the motor memory.

Like many 6-year-olds with a MacGyver mullet, I learned how to ride a bike when I was really young. I had learned a life skill and I was really proud of it. Everything changed though when my friend Barney called me 25 years later. Where I work, the welders are geniuses and they like to play jokes on the engineers. He had a challenge for me. He had built a special bicycle and he wanted me to try to ride it. He had only changed one thing. When he turned the handlebar to the left, the the wheel goes to the right. When you turn it to the right, the wheel goes to the left. I thought this would be easy, so I hopped on the bike ready to demonstrate how quickly I could conquer this. And here he is, ladies and gentlemen, Mr.

Justin Sandlin, first attempt riding the bicycle.

All right, so the faster I go, the better.

Yeah, yeah. I couldn't do it.

You can see that I'm laughing, but I'm actually really frustrated. In this moment, I had a really deep revelation. My thinking was in a rut. This bike revealed a very deep truth to me.

So this was a very, very upsetting thing because this is an engineer, he's very smart, he's figuring out all this stuff and he can't ride this bike. So he then decided he was going to master the backwards bike and 8 months later, Here's the next video.

Practiced about 5 minutes every day. My neighbors made fun of me. I had many wrecks, but after 8 months, this happened. One day I couldn't ride the bike, and the next day I could. It was like I could feel some kind of pathway in my brain that was now unlocked. It was really weird though. It's like there's this trail in my brain, but if I wasn't paying close enough attention to it, my brain would easily lose that neural path and jump back onto the old road it was more familiar with. Any small distractions at all, like a cell phone ringing in my pocket, would instantly throw my brain back to the old control algorithm and I would wreck. But at least I could ride it.

He then got these mechanics to build a small bicycle for his son who had just learned to ride a regular bicycle. And a week and a half later, his kid was riding the backwards bicycle. No problem at all. So that's interesting. So there's a motor memory. Now let's look at a sensory example of something that is similarly disturbing. This is a movie that's being narrated by a guy named Gregory, a well-known psychologist. He's now passed away from the University of Bristol. And if you listen to what he says, he'll make the point quite clearly. Clearly about how this particular problem arises.

This is the hollow head. Actually, at the moment, it's a perfectly normal head of Charlie Chaplin, but wait, as it comes round, you'll see, or will you, that it's hollow. The back of it, coming round now, is actually a hollow mask. It appears to rotate in the opposite direction, and amazingly, the nose sticks out, although it's actually sticking in. Coming round now is the normal, correct, as it were, face. And wait again as it comes round, and you'll see this extraordinary thing, like Jekyll and Hyde. Both the noses stick out because it's so unlikely that a nose sticks in, that a face is hollow. So you see it as convex, although it's in fact concave. As now, and then it'll become the normal face again.

There.

And note that as soon as the features appear in the hollow inside, it will look convex as though it's a normal face almost, though it isn't. As soon as the features appear. There. Your brain refuses to see it as hollow simply because it is so unlikely. And this demonstrates the immense power of top-down knowledge which can actually counter signals bottom-up from the senses and force an extraordinary illusion in which the sensory information of the present is cancelled by immense knowledge derived from the past because you've seen so many faces, all with their noses sticking out. So it's just impossible to see that as correctly hollow.

Okay, impressive. You can't break that illusion. This last one is, for me, the most disturbing, but it's a little more subtle. It's called the McGurk effect. Maybe you've heard of it before. And the person who's going to be demonstrating it is actually a magician. But there's no magic here. It's just. I'll let him explain. You just watch it and you'll. We'll talk about it after.

Ba ba ba ba ba ba ba ba Ba ba.

I just want to say one thing. What you just heard, ba ba, ba ba, ba ba, everybody heard that? That's all you're going to hear in this video despite what you think. That's all you're going to hear. So we'll start again.

Ba ba, ba ba, ba ba. ba ba ba ba ba ba ba ba ba ba ba ba ba ba.

Just close your eyes if you doubt it.

Ba ba ba ba ba ba.

What? You hear. That? Get that? Disturbing. What you. You not only can't trust what you see, You can't trust what you hear. You can't trust anything. It's all— yeah, very disturbing. So these 3 examples suggest that in some way we become hardwired. And— but this hardwired is not the hardwiring of an animal whose brain is built largely based on some program that makes all animals identical. This is experience-based hardwired connections. You don't have a gene for translating those 3 sounds. That's just because you keep hearing people when they use their mouth in those positions, they make particular sounds, and so you interpret them that way. So hardwired implies that it's in the wiring diagram of the nervous system. I'm not going to give you a long rationale for why that's reasonable, but I think if there is experience-based changes to the wiring diagram, and I will try to give you some evidence for that, you have to think of then that the wiring diagram, maybe the connectome, the connectional map of cells, has two roles. And one of them is the innate behaviors that we come into the world with. That have nothing to do with learning. The fact that when you tap your patellar tendon and your leg kicks is a wiring diagram phenomenon. It's based on that, but it's not something that came through— that kind of reflex came through genetics more than probably through learning. But there's a whole second set of behaviors, and for humans this is definitely the most important part, our behavioral repertoire is much more based on what we learn than what we have innately. And these learned behaviors require sensory experience or motor practice or both to instantiate now a hardwired reflexive type response. It's like a reflex, that is, once it's in there, you're not thinking how to ride a bicycle anymore, it's reflexive. But it was learned. It didn't come in through innate programs. And it's this latter type of behavior that humans are more dependent on than any other animal. And it is probably our special trait that allows human cultural evolution to work so well because humans come into the world perhaps knowing less about the world than many other animals, but they have much more time to accumulate information about the world they find themselves in. But there's an unanswered question here, of course, and that is in what form are learned behaviors encoded in our brain's wiring? Nobody really knows. And I want to now step back to the very earliest days of my own research career when I was a graduate student with a scientist named Dale Purvis, who some of you may know, who was at the time in Washington University in St. Louis, and he had suggested that I look at a particular part of the nervous system that had not been studied previously. And it was a part of the nervous system that when I explained it to my parents, they laughed because they thought this was the silliest thing a person could possibly be doing a PhD on. And it was the rat submandibular ganglion. That's the part of the nervous system that allows a rat to secrete saliva into its mouth. And I remember my mother asking me, why would you care? About the regulation of saliva secretion in a rat. She did— do they spit? Do they do something with their saliva? I said, no, no, no, no, no, this is a really potentially interesting place to study things, but it's not because of what it does, it's because it's so accessible and easily understood. So what you see here in this picture, this is a part of the nervous system that's in the periphery. It's not in the central nervous system, it's in the peripheral nervous system, and it's called parasympathetic. That's a part of the nervous system that has a lot to do with eating and digesting food, as opposed to the sympathetic nervous system, which sounds like the part that should be even calmer. But the sympathetic nervous system has a lot to do with fighting and running away, being frightened. That's the sympathetic nervous system. The parasympathetic one is the one that is resting and digesting. Basically. And this is a cluster of nerve cells here that activate cells in the salivary glands to cause them to secrete saliva into these ducts that go into the mouth. And it's this cluster of nerve cells— these are neurons that are so beautiful and easy to get at because they sit in a very thin connective tissue sheet between some salivary ducts and a nerve, this thing here, the lingual nerve, this little triangle, which I'll show you in a second. And I did my PhD on this. And my PhD is published with my name alone. My mentor Dale Purvis did not put his name on my thesis. He claimed it wasn't because he was not happy with it, but because he really didn't do any of it, and he was a remarkable scientist. I helped him with a number of studies during my PhD, but it was a very unusual situation. Nowadays it's very hard to find a student in a laboratory doing graduate work where the mentor says, "You did it. I don't need to have my name on it." So I published my papers in a journal called the Journal of Physiology. I was very proud of them, and I recently reread them, and they They still stand up a little. I'm quite proud of this work. So— but this is 1977, 1980. I dare say some of you, many of you were not even born yet. Is that— ah. Well, I was already your age at that point, at least. So this little area here is a bunch of neurons that are in little clusters, sort of like grape-like clusters. And they receive input from the brainstem through axons that go in the lingual nerve and then touch these cells and make synapses on them. And if we just take a little region like that and zoom up on it and stain the nerves, we can stain the axons coming in and making synapses on these. I used a very old-fashioned stain that has zinc and iodide and osmium in it called the zinc iodide osmium stain. I could see how these cells are innervated, how their axons of the brainstem contact these cells. These cells turned out not to have dendrites. They had a cell body and an axon, so all the synapses that were trying to activate these cells to cause these cells to activate these cells down here, all those synapses are right on the cell bodies. So this is what that little cluster of nerve cells is. Each of these cells has an axon but the label I had didn't label the axons of the ganglion cells. It only labeled the axons coming from the brainstem. And you see this sort of cluster, this small cluster of dots attached with little wires on each of these cells, and those are the synapses made by the nerve cells from the brainstem that are activating these cells. So these are excitatory synapses that cause these cells to be activated. And when I saw this, I had remembered a paper written by Steve Kuffler and Jack McMahon which had a picture in it that said a neuromuscular junction where a nerve comes in and activates a muscle fiber is a very small site on a very long muscle fiber. And that there are these autonomic cells— they were talking about frog cells, but this seemed very similar to me— where there's a bunch of synapses right on the cell body. And I said, oh, maybe these cells are like this and therefore like neuromuscular junctions in the sense that there's only one axon making all those synapses. You can't tell by looking here, but that's what this suggested. So I stuck an electrode in one of these cells and then I gradually increased the strength of stimulus to the nerves that are contacting these cells and discovered that indeed many of these cells, the majority of them, are contacted by one axon only. And the way this is done doesn't really matter. It's in the refractory period of a directly elicited action potential, so I could see the shape of the action— of the synaptic potential without an action potential. None of that really matters here. What matters is that when you stimulate the incoming nerve, this cell responds in an all-or-none way with a big depolarization that could bring the cell to threshold if you were not sitting right in a period here where all the sodium channels are inactivated. That's maybe more for an aficionado, but this shows us the shape of one of those potentials. So, one powerful input to each ganglion cell, Truth be told, about 20% of the cells also had a second input, but the second input was always below threshold. One input was capable of driving each of these cells to threshold, and that's very much like a neuromuscular junction where one axon is activated and the muscle fiber contracts. I then did this same stain in a baby, in a one-day-old rat pup and I saw a staining pattern that at first was very puzzling to me. There are lots more axons but many fewer synapses. There's sort of a paradox. There was more axons around but very few synapses. And when I recorded from these cells and did the same trick here, I saw a result that was very different. Now as I gradually increase the strength to that bundle of axons, any one of these cells would have multiple steps in the synaptic response. One axon after another recruited sitting on top of the previous one, and this particular cell had 5 separate steps, meaning there were 5 different axons contacting the cell, but when you looked at the amplitude of these synaptic potentials, they were much smaller than here. This is probably due to 80 or 90 synaptic release sites. Each of these is a synapse. This is probably due to just 1. So you might have 5 or 6 synapses from 5 or 6 different axons here, Whereas here you only have one axon. And that would explain why there's many more axons but many fewer synapses. Everybody get that? This is a little— this is about as hard as it's going to get. It should get easier after this. Do you understand this? What are you puzzled by? It makes sense. It does make sense. Anybody puzzled at this point? I'm going to try to build on this particular argument. So multiple axonal inputs but each weak. And that raises the question, how do you go from here to here? One possibility is that all of these axons are pruned away but one and that all of these cells are contacted by the same axon. One axon has all of these cells. Another possibility is one axon takes over one part of the territory and another axon takes over another part of the territory. And a third possibility is that each cell independently gets its own axon. And I wanted to figure that out. And so I used the technique of looking at these cells with a kind of microscope that shows them quite beautifully. This is called differential interference contrast microscopy. And I put electrodes in pairs of cells at a time to see whether they were activatable by the same axon. So for example, cell 1 and 2 are shown here as traces. This is time in this dimension and the voltage in this dimension. And I stimulated that bundle of nerves and I could never get one of these cells to fire an action potential without the other one also firing. If I lowered the strength, neither of them fired. If I fired them together, they both fired. And that meant that these two cells, 1 and 2, must be contacted by the same axon because I could never get one to fire without the other. Once I knew that, I took the electrode out of cell 2 and I put that electrode in cell 3 and did the same thing by comparing cell 1 and cell 3 and found the same thing, that cell 1 and cell 3 were impossible to separate as separate from each other, and therefore I inferred that cell 1 and cell 3 were contacted by the same axon, and I knew cell 1 and cell 2 were contacted by the same axon, therefore I knew all 3 of these cells were contacted by the same axon. I then went on to cell 4, and cell 4 I saw the opposite. At one stimulus strength I could get cell 1 to fire, but cell 4 was flat. Cell 3 did not respond. And then if I increased the strength more, Cell 4 fired, the axon that activated it fired, but that did not change the shape of the action potential in Cell 1. So I knew that Cell 1 and Cell 4 were contacted by different cells even though these two cells are actually closer together than this cell is to Cell 1 and Cell 3. I went to cell 5 and found that cell 5 was also contacted by the same axon as cell 1. So these 4 cells were contacted by one axon and this one by another one. And then I could go on and do this for every other cell here and map out all the cells. So I did this. And here is an example of one of these experiments where the same region I am now showing the cells that are innervated by each of these axons, A through F, and these asterisks are inputs that are weak. They're secondary inputs that are not strong enough to drive the cell to threshold. These cells have an additional strong input. So from this, I concluded that the 6 axons distributed their innervation to cells rather than to regions, to cells. You know, it wasn't like one region is activated by one cell. It was sort of arbitrary. These two cells were activated by cell D, these two by axon— by cell E. And multiple synapses of each individual axon clustered onto some cells while seemingly avoiding others in what seemed to be an arbitrary way. There was no rhyme or reason to this pattern. It just didn't look very special. And then I went and did this for a whole bunch of other groups of submandibular ganglion cells, and each shading is a different axon, and every example was different. They were mixed up. The cells were strongly contacted by single axons, but they could be surrounded by cells contacted by other axons, but somehow At the end of this process, when there's only one left, that one has taken that territory entirely and the other axons are not contacting that cell anymore but presumably other nearby cells. So the result implied that axons establish synapses or maintain synapses because there's some change going on in development I've already mentioned based on the identity of the target cell. Even though these target cells are thought to be the same type, it's not like they're doing something different, they're all going to these glands and causing saliva secretion, even though the target cells are of the same type, they choose a subset, an axon chooses a subset that they own, that are their property and no one else's. They don't share them anymore. And could this be some kind of experience-mediated mechanism that would cause axons to choose a subset of subset of the cells when all of them were contacted by each axon to begin with. And this was a connectomic study, but it was done a long, long time ago. And interesting because that idea has stayed in my head. And this brings me to a French term that some of you may know, idée fixe, which is the idea— you know this term? You ever heard this term? It's a French term. Balzac, I think, and maybe Bartók. Berlioz. It's an idea that once an idea gets into your person's head, it kind of dominates one's mind for a prolonged period. It becomes an obsession. I would say learning how to ride a bicycle a particular way is is an idea that dominates your bicycle riding. Learning how the shape of the mouth is relative to a sound you hear is another kind of domination. And for me, those experiments I just told you about took place when I was young and impressionable, and they made a deep impression on me. And sad fact is that I've never shaken the feeling that this is a fundamental insight. Now whether it is a fundamental insight, I don't know. I'm sure it is, but I'm not a reliable narrator here because I'm obsessed with this particular idea. And it's because it was like when you learn to ride a bicycle. It shaped my wiring diagram probably by the very same mechanisms that are going on in the peripheral nervous system. And I'm kind of stuck with that particular configuration of ideas. And so I have a fixed idea about what's important. I believe it in my heart that this is the way it is, and I'm going to give you lots of evidence to support this general idea. But the truth of the matter is it's because this experience happened to me when I was young, and nothing I have done since has dissuaded me of the power of that approach. So let me give you a more modern view of this, that connectomics is a lot easier without pairs of electrodes and more informative if you can get anatomical evidence for what you're trying to show. And a number of years ago, Joshua Elkington and I— Goping Feng was a postdoc at that time— built a bunch of mice called XFP mice. That expressed fluorescent proteins in neurons. And in many of those mice, such a small number of neurons were expressing the fluorescence, you could trace single axons over long distances. And what I'm going to show you now is this particular axon, which is coming from the brainstem and ending up in, of all places, the submandibular ganglion. So we're going to start here. It's one axon. There's a bundle of axons. This is one of those axons that I would have activated in those experiments 39 years ago, but this was done in the new millennium. And then we'll just follow this axon as it gets to the ganglion. It's just one axon. Only one was labeled in this mouse. And at this point, that axon bifurcates. Don't ask me what this is. There is some thing that labels in these transgenic animals that look like that. It doesn't actually look like anything I know. It doesn't look like a neuron. I don't know what it is, but forget about that. Well, of course, if I say forget about it, all you're going to do is stare at it. But— so this axon has decided to branch here. Branching is really important because it's costly to send two branches in the same direction. So you're hoping, at least I would, that very quickly these two axon branches are going to go in different directions. But that's not what happened. They follow each other for a very long, long distance. No more branches. They just keep following each other. This is a total waste. You just send one axon and then branch close. Why you send two axons? And now something up here is happening. So right up here is where the cluster of nerve cells, the submandibular ganglion, exists. It comes in and all of these— everything you see here are the branches of that one axon. And you're starting to see here these clusters of synapses that look like they're surrounding perhaps single nerve cells, and indeed they are. Those are submandibular ganglion cells. We'll go up a little higher. A whole bunch of cells are driven by this one axon. Look at this tangle of tangle. Go up even more and there are a couple very fine side branches. I think these may be what cause those weak connections. Just huge numbers of synapses on cells. And then what's going on up here? There's just one cell up there. The nerve goes twice up there but only one of those branches makes synapses and others make very weak connections on probably cells. So this was anatomical evidence of selective clustering of synapses by individual axons onto some target cells in the submandibular ganglion. Because I could use transmitted light, I could show that there were other cells here. So exactly what I showed you with the recordings, the anatomy shows you instantly. It's very quick to see that, again, it's very arbitrary. There are cells everywhere here. And this one axon has chosen a very small subset. So that's that result. And when I did the same anatomical study at birth, the wiring diagram looked completely different. One axon again comes into the ganglion, but now there are no clusters. And the axon is everywhere. It's just made billions of little branches. They're all there. And it hasn't chosen to put a lot of synapses in any one place. It's just making little synapses on virtually every target cell, and that must be true for every axon. So axons become selective by pruning their synaptic connections with some ganglion cells while elaborating many more synapses on the ganglion cells they remain in contact with. Now, you may think this is something very special about the peripheral nervous system, I just want to show you a result for climbing fibers in the cerebellum that should look very familiar. This is Sugihara's work where he labeled single axons that are climbing fibers that innervate a kind of cell in the cerebellum called Purkinje cells. This is not the peripheral nervous system. This is the central nervous system. And this cell innervates a Purkinje cell here, a Purkinje cell there, a Purkinje cell there, a Purkinje cell there, a Purkinje cell there, a Purkinje cell there. There are Purkinje cells everywhere those dotted lines are. This cell has chosen to innervate a very sparse subset of them in the adult. In the baby, it just goes in and makes huge numbers of connections on many, many cells. So this is not a peripheral nervous system issue. It's an issue that takes place everywhere. Everybody clear so far? Any questions? The problem with all these studies I've shown you is that these anatomical studies are looking at one axon. And you really can't fully understand how this process is going on until you can see many axons at the same time. The problem is if they're all labeled as these are or as they were in that previous pictures I showed you, all yellow or in this case all dark blue, dark brown, you can't discriminate them. And this was one of the motivations for us making these mice called brainbow mice where every nerve cell has a unique color. I'm not going to go through the details of how these mice are made. It's just combinations of fluorescent proteins that allow each wiring diagram of each axon to be visualized at the same time because each one has a different color. So instead of only looking at one axon, you could look at many. This is a picture of the dentate gyrus of the hippocampus, but we use this particularly to advantage in the motor neurons that go to muscle. So motor neurons are in the ventral part of the spinal cord. There's a cross-section of the spinal cord, and these neurons leave the spinal cord in something called the ventral nerve, and that nerve is shown here, and every one of those axons is colored And it's very impressive. You can follow them extremely long distances by virtue of their color. And when I saw this, I thought we had really done something profound until a computer scientist friend of mine reminded me that computer scientists have known this for years. I mean, obviously, you just put color in a ribbon cable and you can follow wires for a very long time. So this is just exactly turning a mouse into a computer wire. But this is interesting. It's a form of convergent evolution where our mammal animal is beginning to look more and more like a computer. Okay. If you look at this at higher power, you can get a sense of just the beauty of the colors. I mean, it's just really beautiful ability to trace these axons. Each one has a very nice color. And then if you trace them all the way to muscle, you now cannot just see one axon's distribution, but now you can see many axons at the same time. So this is a little piece of a muscle with this brainbow transgenic approach where each axon is labeled a different color. These things that are— also look like they're clasping, they're not clasping neurons in this case, they're clasping muscle fibers which are running up and down their cylinders— and these are the synapses axons make on individual muscle fibers. In most cases, there's only one neuromuscular junction per muscle fiber, and you can see by the color, there's one axon going to each of those neuromuscular junctions. You also will notice that two of them are the same color, and that's because they're branches of the same axon. And these two green ones are the same color because they're branches of the same axon. In this way, one can actually trace out all the branches of all the axons at the same time. So there is definitely arbitrarily— arbitrary clustering in the neuromuscular system as well. So let me just show you an adult muscle. This is Ju Liu's work where he traced out all the branches of one axon going to a muscle And then the very same muscle, he traced out all the branches of another axon. Each of these little things is one of those neuromuscular junctions I rotated for you in that previous image, an area about that size. Here's a third axon in the same muscle. And a fourth. A fifth. I know you're getting worried how long is this going to go on, but there are only 11, axons here. So we're almost done. It's just that Julu said it took him so long to do this kind of reconstruction that I basically have a contract with him. He's far long gone from the lab that I have to show them coming in one at a time because if I show them all at once, you won't appreciate how much incredibly hard work he did for his PhD. So I'm putting in ones now that have relatively fewer numbers of neuromuscular junctions, and that is part of something called the size principle. Some have lots, some have fewer. And you see that each of these axons are kind of distributed in some weird way that doesn't make a whole lot of sense. It's not something you immediately understand what the nervous system is trying to convey here. You also see some weird little loops and that's the last axon. So that's the connectome of this little interscutellaris muscle which pulls the ears back in a mouse. And after he did it, I said, great, now do it again. Let's do a couple and see what— how similar they are. Is this some very special program which is organized in this way or is every instantiation unique? I'm going to show you the evidence, but every one is different. Every single one is different. And a lot of it is just crazily suboptimal. It was not engineered by any human. It was engineered without much concern for the fact that you waste a lot of cytoplasm by branching prematurely or by making useless loops. Let me give you one example of this. So here is one of the ones that he reconstructed, and then I asked him just to make everything the same color except one axon that was interesting, which is this red axon here. And this little region here, I'm just going to zoom up right here. So all of the axons go this way except two, this green axon and this red axon. This green axon actually is bifurcating. It's going both ways. But this red axon is the only axon in this muscle that only takes a left turn down here, which is fine. Who am I to judge? It doesn't seem like a bad idea, except that if you follow it up here, it makes a hairpin turn and it goes back to here. I mean, there's a faster way to get there. And then all of this is for this branch here, which goes to one neuromuscular junction. One neuromuscular junction out here is what that coming back down does. You could have done that a lot easier. You, you could have done it that way, but no. But that wasn't all. This axon has a bifurcation right here into two branches, and it has a neuromuscular junction here. That seems reasonable, but that's not what happens. Both branches co-fasciculate up here, and they keep fasciculating, and then one peels off crosses over itself to make that neuromuscular junction. This kind of thing happened a lot and it suggested that whatever is going on in development, it's not aiming to find the optimal solution in terms of structure. It's just trying to do something but it's not so clear what. Because the animal, a mouse, is so bilaterally symmetric, one of Julu's experiments was to and right wiring diagram of the same animal. And they were different. Here's the largest axon on the left and right side, the second to largest, the third to largest. There was a different number of axons. One had 14, the other had 15 axons. The smallest one had 3 branches on one side. It only had 1 branch on the other side. You see there's nothing fundamentally the same about these. There is a tendency for big ones to get to small ones. Small ones and the distribution of sizes is roughly the same, but every detail is unique and this is the same animal, just the left ear and the right ear. So where does all this arbitrary variability come from? And the answer you may have already surmised is it comes from development. Just like in the submandibular ganglion and in the cerebellum, When you look at a motor axon in the adult, you see this sparse, highly concentrated set of synapses on a small number of neuromuscular junctions where each of these gray sites is a different muscle fiber synapse site. This axon contacts maybe 5% of them. You look at an axon at birth, it's a completely different story. In motor axons, they occupy most almost 80-85% of the muscle fibers are driven by this one axon. So there's a dramatic pruning of branches in early life. And if you count the number of muscle fibers that are being activated by an axon, the peak turned out to be right at birth, and then there's a precipitous fall in the number of branches. And in a mouse, by postnatal day 13 or so, you're down to the adult levels. So when it comes to the brain's wiring, it's kind of all downhill after birth. The brain is not getting more complicated, it's getting simpler. This is a little disturbing to me because I think as an adult I'm much more complicated and sophisticated than a child, but I may think that because I've lost the ability to see all the complexity in the world because I now know what's right and wrong because I've pruned away a vast amount of my wiring diagram. So I think this is the general idea that if this is the case, if each axon at birth is doing something like this, then that must mean many, many different axons are going to that site because the total number of neuromuscular junctions at birth is the same as it is at 2 weeks of age. So if you go to a single neuromuscular junction and you put it in the electron microscope and you cut it into very thin sections and you align those sections, you can see how many axons go to one neuromuscular junction on one muscle fiber. Juan Carlos Tapia did this And this is one neuromuscular junction at birth. And you will see there's not one axon there, but just like in the submandibular ganglion and in the cerebellum, a whole bunch of axons all go to the same place, and they seem to be quite happy. They're just rolling on top of each other. There's no glia in between them, and there is, in this case, 11 different axons all going to the same neuromuscular junction. Some have a lot of territory, some have a little, but we're now not talking about the whole muscle, we're just talking about one muscle fiber, and one of them already looks like it's on the way out. If you wait from birth to 1 week of age, it's— things are much simpler. Now neuromuscular junctions only have typically 2 axons, and they square off. One has the left side and one has the on the right side. And then we— because these are fluorescent proteins, we can come and look at what happens over time to a neuromuscular junction at this stage. So here's a neuromuscular junction at postnatal day 11 with one axon having this territory, another axon having this territory. We know that by postnatal day 14, there will only be one axon left. So what Mark Walsh did when he was a graduate student is he took a picture, sewed up the wound, came back a day later, found the same neuromuscular junction. This is in the sternomastoid, a different muscle in the mouse. And at postnatal day 12, if you just go back here, the yellow axon has gained territory and the blue axon has retreated. And at postnatal day 13, he did the same thing. The yellow axon has gained more territory and the blue axon has retreated. Postnatal day 14, now you are down to one axon. And postnatal day 15, that axon has plumped out, sitting on top of the receptors, the nicotinic receptors at the neuromuscular junction, and the other axon has pulled away. Jean-Pierre Changeur, is the first person who I'd heard of who had ever studied this. He's actually sitting here today. This idea that there is a kind of competition between axons at the neuromuscular junction. But seeing this was quite impressive to me because I knew that once this decision was made, this was going to be a lifelong decision that this axon is the axon that's going to is going to be talking to this muscle fiber for the rest of the life of the animal. And that decision was being made before my eyes in early postnatal life. I couldn't help but shake the idea that maybe that is what is going on with memory. So to summarize all this, in all those systems, you start out in early life with neurons that try to contact many target cells in— in this case it's motor neurons contacting muscle fibers, but any of those systems it's the same. A lot of convergence of many axons on the same target cells and a lot of divergence, axons that fan out to many different target cells. And then over some developmental period, choices are made that seem to be unique to each and every animal, each individual of each animal where now Axons have pruned branches and then develop more synapses to strongly activate the cells they remain in contact with. We're trying to fully get the wiring diagram of animals at that first stage with serial electron microscopy. I'm not going to talk about this work. It's still very much in progress and I don't have an interesting result. To just give you a sense of the extraordinary complexity of a muscle where every single axon has been reconstructed at P0. So all of this raises, I think, a question of how things like this could be going on in the brain itself, not just in the peripheral nervous system. And a fundamental question might be, could the same kind of invasive branch pruning that's going on in synaptic connections to muscle and autonomic ganglia is also going on everywhere else in the developing brain. And why do it this way? Why, why, why would you want to do this? One theory might be, could it be that we lay down memory traces as we learn about the world, not by making connections, but by sculpting our neural circuitry via the process of synapse elimination in childhood so that what we're left with, what we're left with is ultimately what we know. That you learn to memorize a piece of music not so much by making your fingers go to the right place, but eliminating all the errors. You eliminate the errors and then you can play the piece by rote. What are young children doing during the period of time when they're supposed to be learning about the world? They are exploring in an obsessive way to, I think, learn everything they can about the world they find themselves in. I'll just show you this interesting movie, a sped-up time lapse of 4 hours in over 2 minutes of watching a baby that is exploring its living room. C'est le mensonge dernier de l'homme que tu pourrais être. Mais laisse ouvre un peu plus à chaque paquet que tu t'entêtes et sans gêne tu te défenses. Tu ne comprends pas que tu détruis une belle histoire, je le nierai pas. Par contre, non, qu'il détruise les murs de ton appartement, on pourra te dire l'amour qui meurt un peu plus à chaque instant. Car il semble qu'il m'affecte déjà et l'homme qui ce que tu as n'est plus que discorde, car ce que tu as rime avec regret. Pour ma part, je n'ai que ce qui reste à jamais, car ensemble on rime avec tristesse. Et l'homme que tu as n'est plus que discorde, car ce que tu as rime avec regret. Pour ma part, je n'ai que ce qui reste à jamais. When it gets stuck. What all that twirling, every object— I've looked at this a number of times. There is not an object in that room, including cardboard flaps, telephone wire, that the kid doesn't actually spend time thinking about. So this profound rewiring theory and its implications. Mature circuits of nerve cells, not just muscle, emerge after a period of pruning from much more interconnected nervous system of juveniles. Pruning is mediated by neural activity, sensory experience, and motor practice that causes synapse elimination. That's the theory. We become what we become based on what remains after all the pruning is complete. We start out with the potential to be many things. And we end up as sadly a small subset of these. Experience, even education— I'm a professor, I'm very sensitive to this. In this sense is destructive. And it is true when you try to convince an adult of a point of view, let's say this theory, if they don't happen to share your belief in it, it's hopeless. It's basically hopeless. It's like trying to get them to ride a bicycle that's twisted because their brains have a very different percept of what the world is what it's like. And we as adults are, although we're absolutely sure we're right, we have no idea. And you're seeing it play out in politics right now in a huge way where people seem to see the same events diametrically opposite and you think people must be crazy. They think you're crazy too. I'll just say that much. So I want to give you some more YouTube evidence that this idea may be true, that we're overwired as young children. And one of the pieces of evidence that this overwiring is generic is that young children are hypersensitive to sensory stimuli. You may not know this, that children find grass very bad. Many children find it too stimulating. I'm just going to show you two quick movies about this. So really amazing. I mean, this kid— that's hard to do. That kid will not— and watch this guy's face. He wants to touch the grass, but look what happens after he touches it. It's like he just ate a lemon. But that doesn't stop him from trying again. And then look at what happens to his face the second time. Go ahead, do it. Whoa. So why is that? Grass is not that for an adult. The other thing is that certain things like meaningless sounds, which don't trigger any laughter in humans who are adults, these stimuli will cause children to go into hilarious conniption fits. "Blong." Yeah. You may have seen that. About 80 million people have seen So what about the brain? Is there an equivalent, if any, to the withdrawal and taking over of vacated sites as occurs in the peripheral nervous system? So connectomics is a way you can begin looking at this. And connectomics is basically using serial section electron micrographs to generate data about every single object in in a scene. And this is a small, really small piece of mouse brain from an adult where every single— this is serial section, so we're playing through a stack of images and back up again. So these things that are moving around are not moving in time, they're moving in space. So things that are moving sideways are sort of moving in this block of tissue at a diagonal Objects that stay pretty much in the same place from one image to the next are just moving up and down. And this can be done manually. This is called segmentation by coloring in, and Daniel Berger colored this in. He's extremely good at that. And there is a big red dendrite that has spines on it running right through the middle here, and there's a green dendrite right next to it. So from this you can reconstruct those cells. So this is one of the dendrites with all of its spines and there's another one of another cell. And this is the dendritic spine that sticks out the furthest. I'm going to show— showing you that so you know where you are in the next picture. So those are the two dendrites that are in the center of the volume that I just showed you. But there are lots of other pieces of dendrites that are in this little volume. So here's that red dendrite and green dendrite. Here are all the pieces of other dendrites that are squeezed in. The dendrites are the receptive sides of neurons that are squeezed into the same little region here. The region is defined as an area that goes out to the furthest extent of the dendritic spines of this cell and that cell. That's the dendritic spine that sticks out the furthest. So there's a lot of dendrites there. Of course, there have to be axons making synapses on dendrites. And here are the axons in that volume. I know this doesn't look possible because this seems like it's filling the whole volume and this looks like it's filling the whole volume. But if you look carefully, you know, where there's pieces of dendrite, there aren't pieces of axons. They're next to each other crammed in there and there's also, of course, supporting cells called astrocytes, a glial cell in there as well, and they take up a lot of room. So that's everything. And there, right there is that little dendritic spine. So this is, you know, either depressing or very depressing. I don't know any other way to say this. The amount of stuff in a little piece of brain is way more than you want to know. Way, way more. And And this was a paper we spent 5 years doing this. We itemized every one of those objects. But it is a very small area. But it was possible. It was tractable. And it suggested to us that connectomics, if it could be industrialized, you could do this on a big scale. And so here there were dendrites, axons, and glia, myelinated axons, inhibitory axons, smooth dendrites, spiny dendrites, oligodendrocytes which make myelin, astrocytes, and embarrassingly, a bunch of things that didn't look like anything we could recognize. And some of these now we know what they are. This is a microglial cell, for example. But others of these still remain mysterious and we had more volume. We traced them out further, but they never really told us what they were. They didn't make synapses. They didn't receive synapses. But they grew long distances. So there's still things we don't know. So this was 5 years of work and that's what it was. These are the 2 neurons that made those dendrites. It's pathetic, actually. So I never show it that way. I always show it big. But it really is almost nothing. There were 1,400 different axons in there, 193 different dendrites. Dendrites, fewer numbers of dendrites and axons. Theoretically, this is an interesting question of why they're not equal numbers, and I'm happy to talk about that if anyone is interested. And there were 1,700 synapses, one roughly every cubic micron. If we looked at the red dendrite and the axons that made synapses on it and also are shown as synaptic vesicles, we saw something that surprised us, which was that every one of these axons, the ones that are not blue, the ones that are yellow or green, are actually finding more than one dendritic spine on this particular dendrite. We were struck by that because it didn't seem like that was likely that you would have more than one synapse of the same axon on the same dendrite given all the other dendrites there. Here's a— showing all the axons on that red dendrite that made contact with more than one spine on that dendrite. So these are axons that are driving the target cell with multiple synaptic sites on multiple spines. In the textbooks, they don't talk about this as a sort of fundamental principle, the way the nervous system works. In fact, the idea is you have a lot of spines to collect information from as many different axons as as possible, and yet in this case there were all these cases where the same axon made multiple branches. Now, you might say since we're only looking in this little region, there should be so many more spines from this dendrite than any other, maybe that's the reason. But here is a movie just showing you all the dendritic spines in this volume. And all those gray things are dendritic spines of neurons that are not the red dendrite. So for an axon to consistently find red dendritic spines to make synapses on requires some kind of effort that we showed statistically could not have happened by chance. One in a trillion. It wasn't even close. It's just impossible. And one of the axons in this volume, for example, this one, had 5 separate spines, all these funky orange-colored spines here that were all driven by the same axon. The chance of that happening by chance given all the other spines in the area here was again infinitesimally small. So there is something going on in mouse cortex that makes axons make synapses, not willy-nilly, but if they find a cell they like, they somehow seem to occupy multiple spines. But this is such a small data that it's hard to take this too seriously. So we tried to do something bigger. And this is a piece of human cerebral cortex from a living human being, a 43-year-old patient who has epilepsy. And to get access to the epileptic site in the hippocampus, the neurosurgeons removed a little bit of the temporal lobe about the size of a mouse brain. They gave it to us. And it's 10 million times bigger volume than in that previous paper, so we were interested in it. And this scalloped edge here is the machine we use, a multi-beam scanning electron microscope that allows us to go very fast. I'm not going to talk about the techniques tonight, but it generated for us a dataset that was impressive. So this image is very high resolution, although you can't tell because it's all on one screen here. It's 500,000 pixels in this direction, 2 millimeters, and it's 750,000 pixels in this direction, 3 millimeters. And this data of that one section, which is only 30 nanometers thick, a thousandth as thick as a human hair, is 375 gigabytes. And we section the brain and we have 5,500 sections like that. So that's 2,000 terabytes or 2 petabytes. I'm more in the megabyte era, so 2 billion megabytes. That sounds like a lot. But it is a lot. Although a petabyte isn't what it used to be, I'll tell you, as you'll see when I get to the end of this talk, which is coming up quite soon. So let me just show you a little bit about this data. This data is so large that although we could acquire the images, we needed help. And just as we were starting to figure— try to figure out how to do this, a colleague of mine who works at Google, Viren Jain, called me and said did we have a big project? They would like to do something big. And I said, oh, we have something ridiculously big. And so we partnered with Google to work on this data. So I'm going to show you now an alignment of the stack of images just to give you a sense of what the data looks like here. So this is actually already cutting through the 30-nanometer sections. You'll get a better sense of that as we move in a little bit further. These are nerve cells, these big things. The black outlined objects are myelinated axons. These are the nuclei of nerve cells, and all of this is that gazillion wires that make up the brain. It's just beautiful data because it was taken from a fresh human, dropped immediately into fix. It was very pretty. So here, for example, is a dendrite with a dendritic spine, spine apparatus, synaptic terminal filled with vesicles making a synapse right there. This is a myelinated axon. These things are the energy organelles called mitochondria. Alexandria. Looks quite nice. So we used Daniel Berger and other people to trace out, as I showed you that colored tracing, to generate ground truth, and then Viren Jain's group is in the AI division at Google, the research part of Google, and then they used that to train a convolutional neural net using a new form of convolutional neural net that works quite well on this data, and then segmented this entire dataset. I will show you that in one second. First, before I show you their segmentation, this will just show you all the nerve cells in that volume, and there's no nerve cells in layer 1, but there are blood vessels there, so there's very few nerve cells there— but these layers are the layers of the cerebral cortex, and these long stripes that you saw there were the apical dendrites of cells. So it's the full cortical thickness, and we could also segment out all the myelin. Myelin comes in a very interesting pattern as well in brains. But that's not what's really interesting. What's really interesting here is that once you have trained data, from human classifiers, you can then basically turn this kind of image into something where every single cell— it's like Brainbow, but now at very high resolution where every single cell is labeled a different color. And not only are the cells labeled a different color, but all the neuropil in between is labeled by virtue of these artificial intelligence efforts. And Miguel Genesee, In Varen Jane's group is the person who is really a master at using this approach. So let me show you what that looks like. So we're now going to zoom in again as we play through here, and you'll get a better impression as we zoom back out again at the enormity of what they've done, but every single piece of this nervous system has been labeled in 3D. There's a leaderboard in Google. I didn't know this. When they work on this, DeepMind is number 2 in the leaderboard of using more compute than anybody else in Google except for their group, which is number 1, at least sometimes. So it's an enormously compute-intensive project. And here it gives you a sense of how much— these are the nerve cell bodies— how much wiring was actually done. So once it's in that form, they've made a nice web browser called Neuroglancer. One of the members of this group named Jeremy made this amazing thing. You just click on any one of these cells and it renders in 3D. And I just rendered a bunch of the cells just to give you a sense of this dataset. And synapses— to find every synapse in the volume as well. So each of these double-colored bars is a synapse and the colors is the presynaptic and postsynaptic side, the axon side and the dendrite side of every synapse. And the last version of this was over 99.5% accurate. So we have now not only the ability to render all the cells, but also the circuits because all the synapses are automatically there. So I'm still waiting for the full circuits to be rendered, but I couldn't help myself and I began working on this just to see if there was any arbitrary clustering as we have been talking about. And it didn't take long to find gazillion examples of axons that clearly were not just randomly making synapses, but making synapses in a very specific way. Specific way. So here's one of these cells I clicked on. I traced its axon and its axon made synapses on two dendrites of another— two different cells here. And I'm just going to zoom up on these two areas here where even though it's only making maybe 10 synapses in the volume as a whole, two of those synapses are on this particular dendrite even though there are tons of other dendrites. That same axon makes 2 synapses on this particular dendrite. Here's a more even amazing example where an axon goes 750 microns, almost a millimeter, 3/4 of a millimeter from here to here. I traced that axon. It made a total of 9 synapses total and 2 of them are on the dendrites of one cell. It crossed by millions of dendrites. It made no synapses on them, but it made 2 synapses on one cell. And here's an example again of an axon of an excitatory neuron that interacts with an inhibitory dendrite here and here. Two synapses on that dendrite and in this case, four synapses on that one. So there's plenty, plenty of clustering of synapses. Synapses even though it's making very few synapses. When it decides to make a synapse on a cell, it just makes a whole bunch of them. Is this learning? I don't know, but it sure feels like it to me. So what's next? So Sydney Brenner died in April, great molecular biologist, geneticist. One of his great pieces of work was the structure of the nervous system, the wiring diagram of a a little worm that has 300 nerve cells, Caenorhabditis elegans, C. elegans. And the header on each page of this was "The Mind of a Worm." And that got this field started, I guess. He was the first person to think about doing a whole wiring diagram. What about the mind of a mouse? Could we do a whole mouse wiring diagram? A mouse has got about 100 million neurons, 100 billion synapses, and it's about 500 cubic millimeters, equivalent to a box that's 8 by 8 by 8 millimeters. If we use the same tools that we've been using to do the human data, to do a mouse brain would generate a dataset that is about 1 million terabytes. Which is an exabyte or 1,000 petabytes. And just give you a sense of scale here. The number of neurons in a mouse brain is 100 million. That is 1,000 times bigger than the largest current connectome that's being developed, which is a fruit fly, which is 100,000 cells. And it's 1,000 times smaller than a human brain, which is 100 billion cells. So it's sort of in exponential space halfway between a fly and a human. It's going to require about an exabyte of storage. We think it could be done in 5 years. It will require the kinds of multi-beam microscopes we use. It would require somewhere between 20 and 30 of them. And just buying them and running them would cost about $100 million. This may seem incredibly expensive, but you would be getting everything, the complete wiring diagram of a behaving mammal. And interestingly, on the 16th of April, the Brain Advisory Committee put out its mid-report, halfway between its beginning in 2015 and its end in 2017. To end in 2025, and on page 93— 91 and 92, sorry— was that one of the transformative projects they'd like to see is a whole mouse connectome. And so I and a number of other scientists have been working diligently trying to figure out how, where, who would to be involved in such a project. I'm very interested in your reaction to whether this seems like total folly, a waste of money, or whether you think it would be useful to have the complete brain of a mammal. And that's all I have to say except to say I didn't do any of this work. This was done largely by my lab colleagues, except actually that's not true. I did the very earliest experiments here. I definitely did those. The people in yellow here are people who are working and have worked on the projects I talked about. I have lots of collaborators. This kind of work cannot be done, this large-scale stuff, by single laboratories without help. And these are many of the people who have been very important, especially I want to emphasize Viren Jain's team at Google. And lastly, Unfortunately, this is very expensive work, so there are a lot of grants needed. So that's a lot. Sorry to have taken so long. Happy to answer any questions if you have any.