The Long Story of How Neural Nets Got to Where They Are: A Conversation with Terry Sejnowski
Conversation between Stephen Wolfram and Terry Sejnowski tracing the history and trajectory of neural networks. They discuss definitions of understanding (operational: executable computational language), the diversity and long timescale of neural-net research (many separate pockets from McCulloch & Pitts onward), comparisons to other scientific revolutions (quantum mechanics), and modern developments including large language models and their massive resource costs. The dialogue mixes technical history, personal anecdotes, and speculative remarks about brain vs. computer architectures.
Chapters
Okay. Hi, everyone. Welcome to a discussion I'm going to have here with Terry Sinofsky about the long story of neural nets and how they got to where they are now. So I guess, Terry, I'm curious, to begin with, when did people first realize that there were neurons in the brain?
Well, I think that the idea of a neuron, by the way, as a unit of the brain goes back to Cajal. He was a Spanish. Spanish neuroanatomist around 1900, and he used something called the Golgi stain, which is like a part of a reduced silver method, like for photography, which stained neurons in their entirety, but only like one in a hundred. And so that produced beautiful fills. And he drew them and was spectacularly successful. And at that point, that was considered the unit that was called the neuron doctrine. But. But in fact.
But before that, what did people think the brain was made of? I mean, there must. There were anatomists before Ramali Kajal. So what did they think the brain was made of?
Oh, yeah, yeah. Well, the. I mean, there were theories that go back to Descartes, thought it was a pneumatic, that these, These, They. They knew about these processes, you know, that they look like wires, but they thought they were little tubes that carried fluid. And they are indeed. But not the pressure. The signal was not the pressure. So they knew about them. And I think that they probably had various theories over the years, but I think it wasn't until Galvani and others, again at the early part of the 20th century, who discovered that electricity was being carried by these nerves. And that kind of started the modern era, the electrophysiological approach to understanding the signals.
But so people knew, I mean, people obviously knew from Volta and the frog's legs and things that muscles had to do with electricity. But how did they figure out that things in the brain had to do with electricity?
Well, let's see. I think that, you know, I can't put my finger on a specific experiment, but I. I would say that it was not much of a leap from the stimulating. Actually, they stimulated the nerve to the muscle. Right. So they. They had the idea that this was coming from the brain, signal transported along the nerve. But I think that it wasn't until Hans Berger, probably, who was able to record EEG signals from the scalp and showed that it was modulated by your level of arousal. And what you were thinking that he saw that it was modulated, and that was a very. It was a microvolt electric signal, but that was a clear indication that there was electrical signals. Flying around inside the brain.
So when was that? Like 1920s or something? Or was that.
No, no, like 1910. Something around there. 12.
So it was around the same time as electrocardiography was being invented, is that right?
Well, that was EEG electroencephalography. That's exactly right.
No, but I mean, the heart version of that, the ekg.
Oh, cardiology, yes. Okay. Electrocardiology, yeah, that probably about the same time. I think you needed amplifiers that were sensitive enough, and I think once you have them, you could record all over the place.
Right. So, okay, so people decided there were all these kind of wire like things in the brain. And did people. I mean, after Romani Kajal did people, was there a huge sort of development of people dissecting different parts of the brain and figuring out lots of detail or not so much?
Oh, yeah, that was a big business because there are a lot of parts to the brain and there are many species that you could look into. And Cahal was one of the still the leaders in his lifetime. He. He wrote the book that is still used today because it's not just scientifically interesting, it's artistically interesting because what he would do is to look through his microscope all day. No photography there. That it was all artistic. In the evening, he would sit down and draw what he saw. You know, the shapes and sizes and the way they're connected and they're beautiful. They're just beautiful art. But it's withstood the test of time in that sense that we still refer to it. And I often show one of his pictures at the beginning of my talk, just because it's just a nice way to honor our foundations and also just a very nice, pretty drawing.
But. So why weren't they using cameras, by the way? Because, I mean, photography certainly existed at that time.
Well, you have to talk to Cajal about that. My guess is that it was easier for him to draw than to worry about photography and trying to keep track of things. I think the Doggerotypes, I think, were developed at the time, but those are really required very high illumination. And I'm not sure they were as had the resolution that you needed at that point.
Right.
But.
Okay, so then we're getting 1920s and things like that. There were electromechanical devices that people were building as to do machinery kinds of things. And there was people investigating the brain. When did those connect?
You know, the, the. I would say that the. The foundation of modern electrophysiology was in that period, the 20s and 30s. And I think it was, you Know, the, the what, what would. The, the types of experience people would do would be to stimulate a part of the brain and then to see what would happen. And, and, you know, you could do that in animals and even in some cases in humans. Actually, what was most, most. What we know about localization of function actually came from war wounds. In World War I, where people got shot, part of their brain was, Was taken, you know, was, was like in the back of the brain, right? There's a common. You have a shell that takes away the posterior part of the brain and you're blind. And lo and behold, that that's. They were able to map out the different parts of the cortex. Now, that was anatomy. But the, but, you know, in order to be able to stimulate and to record really required development of other techniques. For example, they had, of all things, a smoked drum that they use for recording electrical signals. Before, you know, the photography was used routinely. I, you know, I have to say that I haven't read that literature in detail, but it was by the 40s, certainly, things were going full blast. People could record. They had a really good idea of what were called action potentials. There's a famous set of three papers that Hodgkin and Huxley published in 1952, which was the. Now a classic explanation for the basis, the ionic basis of the action potential. And so I would say between, you know, 1930 and 1950, that was a golden period where the new techniques were being developed. And by the way, what, you know, as, you know, in science, you know, everything depends on the techniques that are available for the instrumentation. The way that you take the data, analyze it, and that's. That just changes with every generation.
Right, right. But so, okay, so big, big moment for neural nets. 1943, McCulloch and Pitts. And at least that's my impression of the sort of the big moment for neural nets. What was, I mean, McCulloch, what was. Did you. Did. I don't know all the dates. Did you ever know? Did you ever meet McCulloch?
No, I've never met him. I've written an obituary for him, but I've never met him. I know people that have met him. He was at MIT at the time. He, he was a very interesting character. He, he had a medical background, but he decided he wanted to go into understanding the brain as a, a more basic approach. And he was charismatic. He attracted a lot of really interesting people around him, really smart people. Pitts was a, a, a mathematical genius, but he had psychological problems. And, and, but McCulloch sort of was able to buffer him from the vicissitudes of. Of life. And. And together they published this in this paper that, as you say, did have a lot of impact. And what they showed was that what was known about neurons at the time was that that they had a lot of inputs and then they had a single output, which was either a spike or not. So that kind of is like a binary event, although it's not really, because the timing is continuous. But nonetheless, it was a simple output summation, which we now know is nonlinear, but a single output. What they showed was that with that simple McCulloch Pitts unit, you can construct logic, you know, and gates or gates. And. And since you can. From that, you can compute, you know, you do universal computing. They said, well, neurons can universally compute. So that was the thrust. And I think it had an impact primarily on the mathematicians, but it did, in a sense, it introduced the idea that the brain might compute something like a computer, maybe not exactly, but that it was doing what it was doing was computing.
And so, I mean, the origin of that effort, I mean, so McCulloch, I think, had been a psychiatrist and so on, and was, I think, when they wrote that paper in Chicago, and then, I mean, this was. It's a paper that I think, you know, its only references are to, you know, things like, you know, Russell and Whitehead's Principia Mathematica, things like that. I don't think it has any kind of physiological references, so to speak. It presents itself as something a bit like a Turing machine kind of presentation of, you know, this is how we might imagine brains work.
That. That's true. That. That's why I said that it was written for a computational audience or more mathematically oriented audience. But there was another paper he wrote about the same time which was much more influential on the biologists and. And we now call the neuroscientists. That's a relatively recent area, the term neuroscience. But it was. Let's see if I can remember the title. Oh, yeah. What the Frog's Eye Tells the Frog's Brain.
That was the Jerry Letvin thing, right?
That was with Jerry. Oh, yeah, that was later. That's true. But I guess what I'm saying is that his influence on the brain science was really much more. That paper had a much larger influence than.
Because my impression is if I. If I remember the dates, I might have them wrong. I think, like The Jerry Letbin McCulloch Pitts Frog. What the Frog's Eye Tells the Frogs, end of the 50s that's.
It was. It was probably around that, you know, end of the 50s, I think. So maybe. Yeah, I'll have to check. But, but, you know, that was still in the early days, we're talking. No, it had to be earlier than that. Well, we could look it up, but I think it was in the maybe early 50s or 40s.
It's so. But I mean, the history. Because there were other things that had happened right by that time.
You know, Alan Turing's 43 is the date that I think it was.
That's the original McCulloch Pittsbaper.
Oh, oh, you're talking about Letvin. Yeah, no, that's right. Okay. There.
But in. I mean, you know, there was a whole development, as I understand it from, you know, there was, you know, Alan Turing's 1936. You know, this is what a computer might be like. And then the original 1943 McCulloch Pitts paper was sort of leveraging the Turing idea to say, oh, we can make these neuron like things that can be like Turing machines.
Okay, see, I just looked up the date. It's 1940. What was what the frog's eye tells the frog's brain, really? I'm just looking at it and it says, proceedings the IEEE 1940. I think this is the 41. Now it says. No, wait a second.
I don't think that is right.
Oh, oh, that was the page number. Sorry. It's 1959. You're right. You're absolutely right.
Okay, okay, okay.
I stand correct.
The. I think. Yeah, but, but so, I mean, before that time, from, you know, starting in the 40s, I mean, seems like several things were happening. I mean, there were, there was the, you know, on the more kind of artificial device type side. I mean, there was the, the. The McCulloch pits paper. And then, I mean, people like John von Neumann got into the picture. Wiener, I guess Norbert Wiener got into the picture. And I mean, you know, and von Neumann was writing books like, you know, the Computer and the Brain and so on. I mean, what was the. At that point, there seemed to be a whole development of people saying, you know, we're going to make electrical devices and they're going to be like brains. And people started talking about, you know, giant electronic brains and so on. And I, I think, I mean, that was the, you know, in a sense, in some sense, I think neural nets were perhaps the original vision for what computers might be like.
Well, you know, that. That's an interesting observation because if you think about it, computers came around Relatively late.
Right.
Digital computers. And I think that, you know, the way that a lot of people also, you know, were thinking along the analog lines, including Frank Rosenblatt, you know, who came up with Perceptron. So that was an analog device that he actually built it. He physically built an analog device that it was actually a model of a single neuron. A single McCulloch Pitts neuron. No. It's interesting. The interest in building brains or something that works like the brain has waxed and waned for many decades and with a period of about 20, 30 years. And that was a high period in that era, in the 50s, maybe a little before that, and a little after that. That's when 59 actually was also when Rosenblatt published his Perceptron paper.
But I think that, I mean, one famous other data point was The Dartmouth Conference, 1956.
Yes, that was. That was. It's called the birth of Artificial Intelligence. Actually, what it's called is the birth of that word because people were thinking along those lines for decades before that. I mean, it was. It wasn't like it was a great idea suddenly. I think what really kicked it off was by that time, they did have digital computers, and they did have. They were writing programs and. And they were able to write programs that could do interesting things like prove theorems and play games and that sort of thing. And they figured, wow, if we could, we have a computer can prove theorems, which is the highest level of human achievement, you know, cognitive achievement, math, mathematics, then certainly it should be able to do other forms of intelligence and maybe even, you know, build robots and so forth. And that was, you know, it was based on the false intuition that problems like vision and motor control were easy. Why? Because, you know, it's effortless. We don't have to put much cognitive effort into it, and everybody can do it. You don't have to be a genius to be able to. In fact, people who do sports are generally not as intellectual as people who don't. But that era was really kind of an exciting one. But then I would say petered out when they began to realize how difficult the problems were. And despite the fact that an enormous amount of money was spent by the government, DARPA and R& other agencies, the returns were very meager. In the 60s and 70s.
My impression was there were projects like Machine Translation was a big effort that was an almost complete failure at that time.
Well, my favorite story along those lines is that they had a translator through Russian English to Russian, and then Russian back to English. And the phrase that they tried to translate was the spirit was willing, but the flesh is weak. And it came back. It's good wine, but rotten meat.
But how were they even doing those translations? Was this essentially a textual replacement type of approach?
Well, you know, I have no idea what the actual approach was, but I think that they, they must have had some way of thinking about syntax because Chomsky was using that as the basis of language in that era.
I don't think he was that much involved though. I think the machine.
No, no, he wasn't. No, but he, but the people. You can't think about language without thinking about Chomsky. Right. In that era. And if for another reason, then people quickly realize that if you just replace word by word, it's gibberish. Just, you know, you have to have the right word order. Number, number, you know, two. That has to be done somehow. But, but the, but the other thing is that was completely missing and they didn't understand at the time was that more important than even word order is syntax. I'm sorry, Semantics, semantics, meaning. And that's really, I think, why modern neural networks that could do trans language translation are primarily able to do that by understand, not, I shouldn't say understanding, but by being able to put embed sentences and words into a high dimensional space in which neighborhoods of words share semantics.
But it is kind of interesting that, you know, very early attempt in AI was machine translation. And the technology that's now brought us, our friend ChatGPT for example, is that same kind of technology.
That's an interesting parallel. And I hadn't thought about it. I think that there is something magical about language that really influences humans. By the way, I also contributed in a small way back in the 80s I had a program called Net Talk, which was by today's standards a very tiny network. They had about 2, 300 units in it and about 20,000 parameters weights between the units, which. Which by today's standards is so tiny that I, you know, it's embarrassing by comparison, but it's, it was. We trained it to take in a window of seven letters and to assign the sound, the phoneme to the middle letter. And we trained it up on a bunch of words, dictionaries and transcriptions. And it did a credible job. We actually recorded, you know, from a machine that turned. It was deck talk. Yes, I remember that. It was very nice. It turned sounds into the phonemes into sounds. And you could actually hear the thing as it learned and it was, was mesmerizing. In fact, it Got so much publicity that I was on the Today show and demonstrating that talk to the world. And even to this day, people come up to me and say, I listened to that program and it was the first time I knew anything about neural networks. But it really. I can still remember it. It was one of those moments when you suddenly see something you didn't understand.
Well, it seemed to be very convincing in sort of going from babbling like a young child to talking like a more adult character. I mean, that seemed to be.
It did that and we now know why. Okay, so what it did was when it's learning, it picks up the most important or the biggest regularities. And the most regular assignment is verbs versus nouns. Right? And so it would alternate between a consonant and a sound, but it wouldn't all, you know, it would be the wrong sound, but it would be a vowel. That's. Actually, I memorized that from the tapes. And then it went through a period where it got the small words right. And then it went through another period where it got bigger words right. But it would often have one of the sounds wrong. And then eventually you. Right, you could understand it. It sounded just like a little kid talking, which, which you. Which was. And here, let me just explain why that's important. And it's still today when people, you know, critics talk about these networks, they say, oh, they're stochastic parrots, right? There was nothing stochastic about these. They're all deterministic and they're not parrots. Because there's no way that you could train a network to produce. English is so irregular in terms of the sounds and the letters and the correspondences. There are some rules, but there are always exceptions. And if you keep writing them down, you get a book with 300 pages, but in the same architecture with very, very few weights, few tens of thousands. It was both able to pull out the regularities and the exceptions, and it did it by learning algorithm rather than having linguists sit there and write down rules. So it was. And the other thing is that it has to generalize, right, because we can give it new words. Like I remember giving it Jabberwocky. What? You know, this is the famous poem with nonsense words in it was brillig and the slidey toes, the gyre and gimbal in the wave. Right. It did a credible job, but maybe not exactly the way I pronounced it, but it was English level pronunciation that you could understand.
No, you're right. I remember this. A very cool demo. And thank you for publishing the paper about it in my then very young, complex systems job.
It was one of your first issues. Yes.
Right. But let's go back a bit because I'm curious, you know, the whole question of, you know, we had McCulloch Pitts, 1943. This is what a neuron might be. This is what an idealized artificial neuron might be like. And then. But that was more. You can make sort of logic out of these things. You could make a Turing machine type thing out of these things. But then I think there was the question of, well, in brains, one of the important features is they learn. And I guess sometime, what was it? People like Hebb and so on were starting to talk about how those artificial neurons would be able to do learning. But by that point, people had had the idea that the synapses, the connections between neurons were the places where memory lived. Is how do people conclude that? Or when did they conclude that?
Okay, so the synapses go back to Cajal we mentioned earlier. And by the way, it's below the level of light microscope, which is about a half a micron. And so it was only conjectured that that's the junction where the signal goes across. In fact, there was a big disagreement between Cajal and Golgi, who at Golgi thought that there were direct electrical connections between the two neurons. And they, they hated each other. They. They would be at odds with each other scientifically. And ironically, they both won the Nobel Prize together. But they didn't. We're not on speaking terms in retrospect, this is very common in neuroscience, is that they're both right or both wrong, because their electrical connections, they're called, you know, gap junctions, electrical synapses, and there are chemical synapses. And they both are important in different parts of the brain, different neurons. But the idea though, that you have memory, the synapses came a little bit later, and Hebbian plasticity is probably the most famous. But there were people before that, Konorski and others, who had similar ideas. I think that. The synapse has an allure, which I think is a little bit overrated now. You know, there's a lot of them and there's no doubt that they change in strength. You know, neuroscience has really worked out a lot of the details, but it turns out that many other things are also plastic. In particular, the intrinsic properties of neurons. So what are they? So. Well, first there is the threshold excitability that can be varied. There are a lot of ion channels in the different parts of the dendrites. The other receiving area and the spike initiating zone in there, the soma, and all of those are also plastic that they can be modulated up and down. And so now when neuroscientists talk about plasticity, they really are talking about plasticity of the circuit, not just the synapses. And a beautiful example of that is somatic gastric ganglion of the lobster, which has only a few dozen neurons in it. And it controls the stomach, the rhythms of the stomach. And there are two rhythms. They're very slow, like you know, one to 10 seconds. There's a pyloric rhythm and a gastric rhythm. And it turns out there are neuromodulators that modulate the intrinsic properties and the synapse strengths, which can take you from one to the other. Right? So the very same network, you can have two very different behaviors. And in fact, in our brains there are dozens of neuromodulators that do the same thing. So on a shorter timescale, we're talking about seconds, minutes and hours rather than, you know, days and weeks and months.
So, so when these signals are propagating through like the dendrites and so on, that you're saying that there are different, in addition to the kind of the synaptic weights or something of the strength of a connection between neurons, that there are different sort of resistivities or something like that of the different parts of.
The dendritic tree, different voltage dependencies. Different voltage dependencies, which translates into differences in excitability, differences in integration time, time constants, differences in the dynamics. So even in, within the synapse, for example, there are biochemical reactions taking place that keep track of a history of the signals that have been passing through recently. For example, you know, it's called trace. And short term memory is also a term that psychologists call it. And some of those biochemical reactions can modify the size of the synapse and that makes it more permanent, can change some of the, the, the properties of which ions are flowing through and, and how much. For example, calcium ions are very important and, and that can be regulated. Those ion channels are very highly regulated. So the, the, the nature has taken computing down to the molecular level. That's why it's so efficient and it's extremely miniaturized and, and, and, and it, it, it, it, you know, your brain runs on less than 20 watts of power, right? And just think of everything you can do and digital computers can't do.
Right? But, so we're skipping to the future and we need to go back to the past here to get this narrative. But if we think about artificial neural nets which have these weight matrices and biases and so on, is the thing that you're describing in terms of the characteristics of the whole neuron and its whole dendritic tree and so on, is that captured by that or is that not, or is there some extra term that kind of should be in the future of artificial neural nets that captures these kinds of things that you're now finding happen in actual neurons?
Well, it depends on what you're trying to do. Certainly if you're a scientist trying to understand the brain, then yes, all those details are important. But if you're just trying to extract principles, then there are some principles that are absolutely essential. And I think the different, the wide range of timescales. In artificial networks, there's only two timescales. There's the fast inference, which goes up and down very quickly. And then there is the training, the back prop, popular training algorithm for these weights in the network. And, and that's very, very slow. It takes a long time to train these networks up. So that's the kind of, the longer timescale. But in the brain, there's everything in between. You know, there's an incredible range of, there's working memory, there's all sorts of traces that are there that in fact you can show. This is something that psychologists have studied. It's very surprising they can show that. You know, you can look at, you know, a thousand pictures in, in a few hours and then you come back the next day and they'll say, have you seen this before? And, and you are pretty good at saying yes or no. How the heck could you do that? Right? It must be the case. It left a trace. These things left a trace. An enormous amount of information came in. Now you couldn't recall them. I mean, you might be able to recall a few, but you're, but you could. But interestingly, you can just, you know, if you've seen it before. And here's something else which I think is even more amazing. So when I ask you a question, you know within a second whether or not you know the answer, right? You just seem to know that. Well, in, in the traditional computers, you'd have to search through the whole memory bank, right? You'd have to, you'd have to have an enormous, efficient algorithm to do that. And which means that somehow the brain has used all the machinery and all these different timescales in order to solve problems like that one very quickly. And survival depends on it, right?
I mean, you know, but in a neural net, the information is sufficiently distributed, and the operation of the neural net is such that that same phenomenon of, you know, you're not searching through anything, you're just running the neural net. That, that, that happens there too. I mean, it's in a, in a more traditional kind of digital computer metaphor where you're saying, you know, these are things arranged in a file system or a database or something.
Right. Okay, so here's what it's like. And this is an analogy, so it's not perfect. Okay. It also gets to this point that you were talking about in different between logic and what networks are good at. Networks are really good at pattern recognition. In other words, one pass through and you've got an answer. Right. It's inference.
Right.
Okay. So here's, here's what kind of computer the brain is. It's like a digital computer in which the instructions are recognize the object in 100 milliseconds, bang, it does it. Right. That's one instruction.
Yeah.
Or, you know, construct a sentence in one second that you're going to just spew out a bunch of words. Right. And a lot of it's been automatized so that it's very efficient, very fast. And so you're using these macro instructions that basically can solve enormously complex computational problems in the flash. And the reason there was a lot of evolutionary pressure for that, because if you can't recognize a tiger, you are toast.
Right. Well, let's come back. Talking of. Okay, let's go back to the. We were talking about how people kind of got from the McCulloch Pitts idea, which was essentially an idea of make logic out of something that seemed to be like the neurons people saw in brains to then the idea of how would learning work? And that's something, I mean, that has a much more complicated history, as I understand it, in the sense that.
The.
Neural net that is in ChatGPT, for example, is extremely similar to a McCulloch Pitt's neural net. But the method of training is something that took quite a while, and I think you were quite involved in figuring out how that would work.
So you're absolutely right that learning was really the secret sauce that of course, course made neural networks able to solve all these problems. Of course, many other factors go in, you know, computing, certainly. But what's, what's magical about learning? And psychologists have been studying learning for hundreds of years now. So, you know, it's not like a mystery that Cubans learn. But the, the question was, how do they learn? What's the mechanism. And, and, and, and that was something that was, you know, has, has a. We were talking already about Donald Hebb. So let me tell you a story about Donald Hebb which most people, even neuroscientists aren't aware of.
I, I only, I only know about conceptualization. Donald Hebb was a psychologist, is that right?
He was a psychologist and.
He.
Wrote this book Organization of Behavior, which was very influential. Not because of the book of. Because one phrase in the book, which is that if two neurons fire, are activated simultaneously, then the connection between them should be strengthened, as they call the hep synapse. So coincidence of firing leads to increase in strength. That was the idea. And it was in fact when electrophysiologists actually did that experiment. That's what they discovered. Turns out to be much more complicated. But let me tell you the story. I was asked to write a review of that book in 19, was, let's see, oh, probably around 1980 or something, but it was published in 49, so it's many years later. Right. But. So I actually read the book and I always thought that, well, this is an algorithm that is for associative memory because you associate an input with an output and that's kind of the essence of associative memory. More complicated when you have interactions in the network. But no, he hate the behaviorists were very dominant at the time and he hated behaviorism. Behaviorism is the idea that you could do Pavlovian condition, classical conditioning, but also operant conditioning to train animals to do a repetitive task, to train them to recognize objects. And they were using it on humans too. Humans also have classical conditioning. All animals, including flies, invertebrates. But I was completely wrong. Here's, here's what the problem was that he thought he was solving. It's called the persistence of activity. So you know, if you have a feed forward network, what happens is the information flies through and it is gone. But how is it that you can remember something that happened, you know, a few minutes ago or, you know, an hour ago? Right. There has to be some way for the activity that came in to cause some persistence, like for short term memory in particular. And the only way that he could think of doing it is if you had some kind of recurrent connections, you know, that would circulate activity. And then he said, well, how could you learn the new thing? Right? Because you want. It's not just a delay line, right. It's got to be some way that you've strengthened synapses. And so he had the idea that if you strengthen all the synapses in a line, as the activity goes through a bunch of neurons, and especially in a circle, that that would maintain activity that came in for a longer time. And actually that recurrent networks are exactly that. They hold on to information that is given at the beginning of the sequence and they use that to then combine it with information coming in at the end of the sequence like sequences of words. So, so that, that was. And, and he, he did, he didn't say anything about when you decrease the strength of the synapse. Right. Because obviously if it keeps increasing, you're going to saturate. And, and so interesting. What, what, what people discovered was that if this, what's important is not that they come in at the same time, but whether one comes in a little before or after within 10, plus or minus 10 milliseconds. If the presynaptic comes in the input before the output, within a window of 10 milliseconds, and you increase strength. But if the input comes in 10 milliseconds after the output, then you decrease the strength. And if you think about it for a second, there's a. What's implicit here is the concept of causality, because if it comes in before, it could have contributed to the output, but if it came in afterwards, it certainly can't. Right. And that's the essence of persistence, is that you have persistence of a causality chain.
So that, so that's an actual mechanism that's observed in, in neurons.
Yes, that's, that's been, that's been established in the cortex. It's in hippocampus and the superior colliculus. It's, it's, it's a, it's a universal mechanism. It's not the only one, by the way. There are many other forms of plasticity. I, I gave you a couple examples earlier. But even with synapses, for example, in the hippocampus, there's a beautiful paper that was published just a couple years ago, like four or five years ago by Jeff McGee, showing that in hippocampus there's another form of plasticity that has a window of one second. And it's when you're creating what's called a place cell in a rat, when the rat's running around and you stimulate a pathway and suddenly this neuron becomes active every now, whenever the rat comes back to that location, it's called a place cell.
So going back to Hebb. So what you're saying is that Hebb imagined that, that you would learn things by virtue of recirculating that information through this neural net and that if it came through again and it looked like what it was before, then it would be, then that weight would somehow be increased. Was that kind of the idea?
Well, at every stage, at every point, at every point along the chain, you increase the strength. So now you can, it comes back to the beginning and it activates the neuron again. So it's like a chain reaction.
Right.
You know, very simple idea. And, and, and in it, but you know, in, in, in modern networks with a lot more connections, it's not a simple chain, but it has, in a recurrent network you can like, it reverberates, you know.
Right.
An analogy with physics.
But, but so that idea from, from Hebb, did that, did that sort of grow into some big kind of chain of things? Because by the time one was talking about things like the perceptron and Rosenblatt and so on, that wasn't a learning system. That was a, that was a pure.
No, no, but, but, okay, so now here's a really interesting chapter in neural network history, which is not always people aren't aware. I doubt if you're aware, but it's very important. There was a period in the 70s, 60s and 70s, especially in 70s in Japan, when a couple of Japanese started actually building things, networks that were precursors of the ones we have today. For example, we have something called convolutional neural networks, right. And they're used for object recognition and vision feed forward and they have the architecture you see in the visual cortex. Well, Fukushima in Japan, who was working for, I think the television company at the time, had this idea of just building it, right. And even I had the heavy implanticity. Now he didn't get very far because the computers there were incredibly slow. But nonetheless he had the concept and he was given the Franklin Prize a couple years ago to recognize his pioneering work. And another Japanese.
Was that the, the thing. I remember only this one keyword, the neocognitron. Is that.
That's it. That's it. Okay, well that's, that's it. That's the. I was talking about this.
I only got one word, but yes, I, I wasn't.
Well, you have, you have. Half the battle is that you can Google it. You'd find out a lot more. But the other person, the other person, before I go on, is, is Sunichi Amari, who was a mathematician and he had developed networks that are kind of precursors of the Hopfield network. Now they didn't, weren't the same hopfield network is characterized by symmetric connections. But, but he had the idea though that you could, you could get pattern recognition out of them. And then there's a whole family of associative networks that were, came out of Stein book from Germany and others, but those have been superseded in terms of their architectures and the learning algorithms.
But okay, going back again. So there was Hebb. And then the next big thing that seemed to be happening was things like the Perceptron and Frank Rosenblatt and that was what, late 1950s or something?
59 was the period when he had his book. And then that was when he got a lot, you know, it was, it was a competing approach to artificial intelligence. And he, he, you know, he had some demonstrations which were very interesting. He actually built a giant analog computer with potentiometers for weights and, and he, he had visual inputs that he could give it to it. And you know, with, with photodiodes, I give or you know, some kind of.
In modern terms this will be a one level, one layer neural network. I mean, this was a single, this.
Was a, it was one layer of weights, you're right, one layer of weights. And, and, and you know, you, there are a class of problems for which you can actually come up with good solutions that, they're called linear predicates where in the weight space you can draw a parallel plane that goes through the examples. If they are, if the positive and negative examples are on different sides, then you can do a discrimination between them and it will generalize to new inputs.
So in a sense we've gone from the one layer neural net to what ChatGPT is maybe a 400 layer neural net or something. And that's the, well, that's why they're called deep.
So there was a period in history, I remember I told you about the fact that it waxes and wanes, right? So in that era, the early 60s, there was the resurgence of interest in artificial neural networks. FRANK ROSENBLATT ARTIFICIAL intelligence by writing programs for computers and so forth. And Marvin Minsky and Seymour Papert wrote a really sophisticated mathematical monograph called Perceptrons where they did an, a really thorough analysis of this with what I just told you about linear predicates. And you know, and they show that there are a lot of most interesting problems don't fall into that category. So therefore, you know, it's a, it's not a very good starting point. And they, at the end of all this beautiful math, they had this kind of the speculation saying that in their view There would never be a generalization to perceptron learning algorithm. The key was that you could learn it from examples. Right? That's what Rosenblatt really did. Right. He, he added the learning algorithm, which, which was very, was Hebbian actually.
What, what, what was his learning algorithm?
So what, oh, okay, here's what you do. You take the output from, you give it an input. You take the output and, and then you compare it with what it should be. You know, is it part, is it a yes or no? Is this a cat or not? And then if it's right, you don't do anything. And if it's wrong, you use that error to update all the weights so that the next time, you know, the weights that should have pushed it above threshold get stronger, the ones that should have pushed it away get weaker.
How do you know which ones to change?
Oh, you know, there's the gradient. You just see, you just ask how does the error change with the, if we change the weight.
So it's the same story as modern.
You know, it's exactly the same story for back prop, right? It's error gradient. That's all. It's all.
Did Frank Rosenblatt figure that out? Was he the person who figured out.
Yes, that was the beauty. That's the perceptron learning algorithm. He figured out gradient descent. Now, it was a convex problem. That means that the beauty was that didn't matter where you started with the, the weights. You were guaranteed gradient descent would take you to the bottom, which would be the best set of weights that could, you know, to be able to solve that problem if it was solvable.
What was Frank Rosenblatt's background? What, what, how did he get into the psychologist?
Yeah, he was a psychologist, but he had very broad interests and you know, he was interested in first of all, a lot things in, in AI or electrical engineering. And so he had a lab where he did experiments and built things. And yeah, he, he was a polymath, you would call him today.
Right. So this was a time when, when there was early interest in machine vision for military applications, for optical character recognition for things like that, if I understand correctly.
Yeah, well, that's always been, I think, a practical problem. In, in fact, one of the first successes was of, of neural networks was handwritten digit classification.
Right.
For the post office. This is Yann Lecun, by the way, I want to finish the story with Martin Minsky and paper because this, this, this really had a huge impact, right? They, they speculated that there would be no generalization of that learning algorithm to multilayer networks. And that's where Jeffrey Hinton and I started. In other words, if you talk to anybody in AI or engineering, what they would tell you is that, forget it, you're working on something that. Which is a dead end. And we have mathematical proof of that. Right? There was no mathematical proof. It was just buried in this very complex mathematical book. But it was purely speculative. Right. There was no theory behind it. Except. But in any case. So Jeff and I started from a Hopfield network, we heated it up, and we discovered that Boltzmann. We called it the Boltzmann machine because it was fluctuating and you could use physics to understand the equilibrium states. And we showed that in equilibrium, there was a learning algorithm for an arbitrarily as many layers as you want. And in fact, even it's really beautiful because it was a local algorithm, unlike backprop, that requires global information of all the gradients and all the weights. It could all be done with just local correlations between pairs of neurons. So it was Hebbian. Hebbian plasticity bolster machine. We've solved the problem of multilayer learning. We thought we had figured out how the brain works, but, you know, it turned out to be a little bit more complicated.
I remember that paper. Yes, that was. That was. Was like 85 or something. Is that right?
Yeah, exactly. That was. Yeah, that was. Yeah, that. That came out. The learning algorithm came out. The actual. I think we. The first. That was. That was the paper. Cognitive science. Yes, 85.
But. So. So, okay, well, I'm. I'm still. And I know. I mean, back to the. The. I mean, the. The time of perceptrons, right? I mean, Marvin Minsky always used to claim that he didn't mean to kind of kill the field by. By writing that book. Although you must have talked to him about it.
Oh, you know. Well, you know, Marvin, you've talked to him a lot. You know him probably better than I do, but I actually have evidence for this, firsthand evidence. So at the meeting that was held in 2006 at Dartmouth, the 50th anniversary of the 56th meeting on AI, and there were about, oh, I think about six or seven people at the meeting from the original meeting. Right. This is 50 years later. So it's a. These are old men now. Marvin was there, and at the end of the meeting was a banquet. And at the banquet, you know, each one of them got up and said what they thought about the meeting. It was. It was a very interesting meeting. Looking forward and backwards, you know, the future of AI and where are we now? And so they asked her questions. So I got up and I asked Marvin a question. I said, well, there are people in neural networks that consider you to be the devil because of the fact that you held back the field for decades. Are you the devil?
Okay.
And this, this, this, this, this is. I've never seen this before. It's like, you know, I, I, I, I just, you know, suddenly he just got extremely animated and he just launched into this long tirade. You know, it was going on and on about we didn't understand anything about scaling or complexity and blah, blah, blah, that, you know, parody, it doesn't do parody and so forth. And he, and, you know, it was, it was all, you know, it was, it was clearly. I, I had pressed his button and so I finally had to stop him because, you know, it was, it was getting embarrassing. And I said, Dr. Minsky, I asked you a binary question. Yes or no? Are you the devil? And he kind of spluttered. He kind of spluttered for, you know, he didn't know what to say. And he finally said, yes, I'm the devil. Okay, okay, okay.
That's very Marvin esque. I don't know. Interesting.
Now, I've come to actually moderate it in the sense that I don't think his was the only reason. I think that may have contributed, but I'll tell you, I think the real reason is that in order to make progress, he needed a lot more computing. They didn't have it back then, but.
You know, one of the ironies, Marvin, when he was a graduate student at Princeton, had actually made a, you know, a physical electronic neural net thing.
Yes.
The Snark or something. Was that what it was called? But I don't quite know what it was.
It sounds like something in Lewis Carroll.
But, yeah, we're right.
It could have been. Yeah, it could have been. No, no, you're, you're, I heard that story and, and I don't, I'm not sure. I don't think it's anything published. But yeah, he, he would build these little networks out of parts, electrical components, and, you know, and, and, you know, I, I wonder about the psychology there, because he clearly must have been very intrigued. And, and, and I wonder if he got burned or something that he turned his back on it. I don't know. But he wrote his thesis on this in the math department, and they didn't know what to make of it because, you know, nobody in the, in the department knew anything about the neural networks. And so they sent it over to the Institute for Vast study, and the mathematicians there only talked to God, right? So they sent back the following cryptic comment that it may not be mathematics, but someday it will be. And they were right. They were absolutely right.
I wonder who wrote that. I think von Neumann, I think, had died by that point. I mean, von Neumann would have been somebody who, I think, you know, because von Neumann was, was pretty involved in this whole, you know, brains versus computers, et cetera, et cetera, et cetera. Although I don't think very much. Yeah, I mean, I'm not sure he ever figured out. I don't think he really did. I mean, he had a whole analysis of probabilistic logics and so on because he was convinced that, you know, sort of error rates were a critical feature of brains versus computers and so on. But I don't think he ever worked on neural nets. I don't think he, I think he, he had read the McCulloch Pitts paper, but I don't think he ever worked on.
Well, he was, he. In his Stillman lectures at Yale, he raised an interesting problem, which is that how is it that you prevent the introduction of imprecision, noise or error from propagating? And he came up with redundancy as one way. But it's still a very interesting question, right? How is it that the brain is so resistant to not, not just a little bit of noise, but also you can scoop out a big piece of the brain, as long as it's not in a critical area that controls, you know, your breathing or something, you're. You're okay.
I'm sure the same would work. I'm sure if you took ChatGPT and you scooped out, you know, a billion weights, I'm sure it would do just fine.
I, I think so too. And that's the point. It was called graceful degradation. And that has, I think, to do more with not so much redundancy being that usually means having identical replicates that then work. And you take the average. I think that it has something to do with multiple pathways, that there are many ways to solve the same problem, that the brain has advantage of, you know, converging input. It's. And also the other thing that is a major difference is that the, the brain has, is probabilistic. It has to keep track of evidence and, and a lot of it comes in from different sensory systems, you know, vision audition and so forth, and has to be combined and, and to come up with the best guess about what was Just said, you know, word. By the way, you know, people who are going deaf, it helps if they're looking at the person's face because they could do lip reading. Right. Because there's a lot of information about the sound from the lips. And so. But how do you combine that? Right. You have to combine those two senses, and you have to use probabilities in order to be able to eventually make the right decision. And by the way, that's the way the transformers work. What they do is they just don't pop out a word because that's the only word, but they assign probabilities to all the words. And the ones that have the highest probabilities are the ones that are used for the next loop, the next word in the subsequent output.
So coming back to the history for a minute, I mean, at some moment, the physicists got involved. So sometime around maybe late 50s, early 60s, there started to be physicists who said, you know, these neural nets are a little bit like spin systems and we can, you know, a little bit like, you know, magnetic, you know, systems of, you know, the, the neurons are a little bit like spins up and down in magnetic systems and things like that. And there can be analysis done that way. At least that's my. I mean, I don't know whether. Whether those people are even. I mean, there were people like, I don't know, it was a fellow can yellow or something who was. I remember running across at some point.
Yes. Yeah. So there's a whole cast of characters from that era. Kayello was one of them. And actually there was another guy who was at Irvine. I've gotten his name now, but Gordon Shaw. Gordon Shaw. You got it. And, and you're right, they had this intuition that, you know, you should be able to use condensed matter physics to understand it. But these, these were isolated people who were publishing in obscure places and I would say had little or no impact because they weren't part of a community. There wasn't a community. It was basically a lot of isolated people, very smart people, but going in different directions. And it wasn't, Wasn't really adding up, you know.
Right.
And. And I, I, you know, and the other thing, of course, is that the neuroscientists couldn't understand anything that they were saying because they didn't have any background. So they didn't have any impact on the neuroscientists. They may have had some impact on some of their colleagues, but their colleagues didn't know anything about the brain, so they couldn't help them.
What problem did they think they were solving?
Okay, I think, in fact, I know Gordon Shaw. I knew him and I talked to him a little bit about this. And he. He really felt that there was some kind of collective phenomenon that was going on in the brain that was. It was like a phase transition or something which allowed many neurons to work together collectively. And he, of course, used analogies from physics as a way of understanding that, like, you know, the magnet, the phase transition, magnetism and so forth. And there's still people who kind of think about those lines of phase transitions. But. But the. Okay. My first experience with this group, by the way, was a meeting that I attended in 1979 here in La Jolla at UCSD that was organized by Jeff Hinton. He was a postdoc here at the time working with Dave Rumelhart and later.
That was a psychology. These were psychologists?
Yes, they were cognitive psychologists. Jay McClellan, Dave Rumelhart, the PDP group was here in. In the 80s. And so he organized a meeting which was called Parallel Models, Associative Memory. And that was probably the most well established neural network model. And the idea was you have inputs and outputs and then you try to store patterns in the weights and you could ask questions like how many patterns can you store and how much interference between them and so forth. And so he invited a bunch of these isolated people, you know, working like Don Geeman from statistics, another guy who was very interested in Kohonen, Teivo Kohonen from Finland, had a way of using collective neural networks to do categorization and others that. Jerry Feldman, who was a computer scientist from Rochester. And, you know, so we're talking about people from many different departments, and I was coming from physics. I was actually a postdoc at the time at Harvard Neurobiology working with Steve Koefler. So I was making a transition to neuroscience.
But you'd already been involved with neural nets by the time. I was just looking. And I found a 1975 thing of yours about neural nets.
You're right about that. I had already gotten bitten by the neural network bug before I got my PhD, and I did that primarily on my own, but I had a lot of help from John Hopfield was at.
Princeton at the time, But John Hopfield was a condensed matter physicist who came out of very much that physics, but a different tradition from the Gordon Shaw world and so on. Is that correct?
Well, he was actually a biophysicist at the time, but, you know, he had done work earlier, work on some condensed matter problems, but.
But no, at that point he was working on more microscopic biophysics.
Oh, things like hemoglobin, you know. I see which, which is an interesting problem. There might be a phase transition there, you know, when you, when you bind oxygen and, and, and when you give it up and, and, and you know, and he had, actually, he had a very influential idea about error correction. So when you're, when, when DNA is being, when RNA is being translated to proteins, if you, if there's a certain error rate, you know, one part in, in, I don't know, a thousand or something, but that's not good enough because, you know, you can get all kinds of mutations then, or in the proteins. And so he realized that you could fix it by error, by error, error correction, but it would take energy and it has to be irreversible, otherwise it could go backwards. And so that turned out to be true. And, and you know, that, that was a very important insight that comes from physics and was. Subsequently experiments were done at Bell Labs, interestingly, where he was a, I think a. He spent, he spent time there as a consultant. He was at Princeton, his academic appointment. But, but, you know, so he was in that kind of tradition thinking about biological problems. And he got bitten by the neuro, by the neuroscience bug. It was going around, A lot of people were infected. I was. And, but, but he, he, unlike me, had access to all these meetings from, you know, because the neuroscientists thought, here's somebody who's really smart, you know, we should invite him. So he went to the, There's a whole series of meetings. The Neuroscience Research Program in Boston. NRP had these meetings where they brought in people from different areas of neuroscience to come together. For example, I still have my shelf one on neural coding. Ted Bullock was the person who put that one together. Very, very prescient. And John would go to these meetings and he'd come back and tell me about them, right. And I would hear all about the, the wonderful, interesting things happening in neuroscience. And it may be even more interested in trying to solve, you know, figure out how the brain works. So that was that, that was, you know, led to for him the 82 pns paper, which was the Hopfield Network, that had a huge influence and, and an influence first on the physicists. I think he like, was the. Able to bring into neuroscience people like Chaim Sam Polinsky now who's major figure in computational neuroscience. You know, there was, Yeah, I remember.
You know, he was at Caltech at that point, and so was I and I heard about that. The you know, that network idea, probably in 1881, something like that. And I was. The frustrating thing for me is that I never managed to actually reproduce his results. That is, I wrote programs that should have done what, what his paper said. You know, the things that his paper said, and they never worked. It never worked.
Stephen, you had a bug in your program because I, it, it works for everybody else. Oh, wait a second. Okay, so there's actually a subtlety here, very interesting subtlety. So it would only work if you did the updates asynchronously.
Yes, yes, yes, I know if you.
Try to do it synchronously, it would, it would go into, you know, limit cycles and a lot of other things. But, but if you did it asynchronously, it was guaranteed, literally mathematically guaranteed to converge by.
But as I recall, I mean, what I tried to do was to map out the, the, you know, it's a. You know, the big idea is you get these attractors where, you know, there'll be a bunch of possible inputs and they all evolve to a single possible output. I mean, that, that, that multiple.
Multiple a local minima that were point attractors and could be used for doing what's, what's called completion. In other words, you give it a half of a vector, which puts you close to the basin, and then it would complete the vector and give you the entire output. So it's completion.
Because I'm a good archivist, I'm sure I have my actual program from that time. Go find out.
No, I'm not accusing you of making a mistake in your program. I'm just saying that you must have had, you know, you putting in the wrong inputs or something. And it could be that you were looking at the wrong output even. But.
Okay, but so, you know, the point there was this was essentially a mathematical structure, not particularly intended at the time, I think, to have direct sort of correspondence with things in the brain, but a mathematical structure which would allow you to do this thing of going from many possible inputs that could represent, I don't know, a letter A or many possible things that might be like a letter A, many possible things that could be like a letter B. And it would go to the actual sort of single point attractor of it's an A or it's a D. And.
The reason why that was an important step forward is that up to that time, the people who are working on these associative networks worked with linear ones, linear summations, and then a threshold. And the hopfield showed that you could do the same thing in a highly nonlinear network. And you could store and you could actually estimate the capacity very nicely. And it had a lot of nice properties. And like I say, it led to the Bolson machine and a really nice learning algorithm. So it was a breakthrough, I think, in conception.
Yeah, right. No, actually, I remember this capacity thing was like the. How many vectors can you store in before the thing, you know, how many attractors fit in your space, so to speak. How many distinct. Yes, that was, that was one of.
The, that was one of the. The. The. They. You could, you can actually analytically show what that was. I think it was like 0, 19 N, where N is the number of neurons.
Right. But so, okay, so this happened and that, I mean, that one feature of that network is it's. It's very recurrent in the sense that you put an input and it keeps on, you know, doing more stuff to the same input. It's not one of these kind of just feed forward ripples through type. Type things.
You know, that is that. So that's what made it impressive, is that he harnessed the nonlinearity to do something interesting.
Right, so, okay, so what happened next? So then that was a single network where you kept on recirculating through the network as opposed to multiple layers of perceptron like network.
Yeah, that's right. And because of the fact that it was kind of limited, you know, there were no hidden units in, was very difficult to build on that. Although a lot of people, I think, were thinking about, for example, I know recently I've gotten very interested in learning sequences. And there were people who would put in asymmetric connections that would allow the network to settle into one local minimum and then jump into another one. Right. So I think even, you know, people you probably know, David Kleinfeld, I think here was one of the people. But, you know, there was a tremendous amount of activity. It was really exciting. And the Neural Information Processing Systems meeting brought together the physicists with the neuroscientists and the computer scientists and the statisticians and people studying computer vision and speech recognition. They all came to these meetings that were organized originally by Posner, Ed Posner at Caltech, but then he died tragically, and I became the president of the foundation that ran it and have been ever since. And what was exciting back then, it was really exciting just to be able to meet people from all those different fields. And they all had the same problem, which was that the tools and techniques that were available at the time didn't allow them to make Progress with vision or speech or robotics, it, with traditional tools that they had. And they, they, they just thought that there had to be another algorithm or architecture that would do that. Neural networks were very promising because there's learning algorithms, right? You might be able to learn the solution. And, and so, and we had a, the problem was that was a, a Tower of Babel because everybody would, would be speaking in their jargon and it was hard for other people to understand what they were saying. You know, the, the, the physicists would talk about, you know, all these equations and attractors and so forth and the neuroscientists couldn't figure out what that was. And, and then the neuroscience would come up and talk about, you know, all the, all of these Greek words for different parts of the brain, the lateral geniculate nucleus. I mean, you know, it was all Greek, you know, impenetrable. And, and then the only ones that actually, I think got through were the engineers because they actually built things and they could demonstrate them. Right. They said, here, look, look what I'm building here, here's what it's supposed to do. You know, they'd have their own language, but they would at least be able to explain, you know, what, what, what the, what the goal was and how they did it.
You didn't mention the mathematicians who I think were not, I mean, you know, the attractor concept was one of the really from mathematicians, not so much from physicists, but I don't think they would.
Well, the mathematicians came to mainly statisticians, but there were some basic mathematicians, dynamical systems people, but not, not as many in retrospect. I think that's one of the things that was missing was that people, I think we didn't appreciate how important dynamics is going to be and understanding the trajectories in these very large scale networks.
Although the truth is dynamics is not, I mean, you know, in a chat, GPT for example, it is a feed forward network. There isn't, you know, there isn't really dynamics.
Okay, okay, well, you know, you'd be surprised. Okay, first of all, there's a loop that takes you from the output to the input and it circulates. That's dynamics, right?
It's a, it's an outer loop, so to speak.
It's an outer loop. But that outer loop is found in the brain too. It's between the cortex and the basal ganglia. Then that's how I'm able to speak to you in a sequence of words because my basal ganglia has learned how to, how to do that. And with automatically, without having to think about it.
Right. Well, so, but, okay, but back. Okay, so we're now in the 80s and we're heading for your contribution with Boltzmann machines and so on, which was. So I think some precursor to that must have been like the simulated annealing idea. I mean, people have been, you know, you're trying to find the minimum of this kind of mountainous surface, and you're using gradient descent, and you typically get stuck in the mountain lakes, so to speak.
Right.
And then there was this idea from what, 1980 or so, of if you kind of just jiggle things around, like you do when you are trying to make something crystallize. Well, in annealing, if you jiggle things around, there's more likelihood that you'll not get stuck.
You just brought back a memory, this vivid memory. So Jeff and I were collaborating at the time, and our view was that if you connected up the network properly with the right connections between different features, like we saw in visual system, that we could make progress with vision. And then the question is, can you learn those connections? And I remember I just had read a paper by Scott Kirkpatrick on simulated kneeling. He was at IBM, Thomas J. Watson at the time. And, And I said to Jeff, you know, we could. He, we, you know, the Hatfield network is running at zero temperature. You always pick the lowest energy. You, you either flip up or. Or not, depending on which has the lowest energy. So you're always going downhill energy. That's why you can prove that it has to converge some eventually. And I said, we just heat it up. And so that was that. That I remember because once, once we had that, then we, we were able to start thinking about, first of all, avoiding local minima at high temperature. So you go to high temperature and then you gradually reduce the temperature. And if you do it slow enough, you can prove that you'll eventually find the global minimum, but you have to do it really slow.
I, I dispute the claim that you can prove that.
Well, it's. You have to go in logarithmic time. That that was. The, the hook was that you have to go really slowly.
Okay, okay.
But, but, but nonetheless. And it became longer and longer, the bigger, the bigger the network, because the whole thing has to come to equilibrium to all the parts and, and you have to trade information, activity patterns.
I see, I see. This is specifically for a neural net, because.
That's right. But, but, but what we, what we discovered, and this is a. For me, it was a shock because that's not what we were looking for. But we discovered that something magical happens in equilibrium which is that suddenly with just Hebbian learning you can reduce a loss function that had to do with the loss between the input and the output. If you're training it to do some kind of categorization or just any function, any, you're going from input to output function, you could minimize that over time. And again, if you come to equilibrium, you can show that if there's a network that is able to do that transformation or categorization, then both of a machine could find it and it could have arbitrary number of hidden layers. It just took enormously a long time for it to come to equilibrium and then compute all the statistics. You have to compute the average correlation over many, many time intervals.
But, but is that happening? Because basically it's ergodic. It will eventually visit every state. And so one of those states.
That's the assumption. That is, that's the assumption you have to make to come to the conclusion that it's going to find the solution. And, and, and you know, it, it just, it's just a very slow process and you know, it, it, it. I still think it's, it's much more beautiful than back prop. Back prop is kind of a statisticians nightmare. You have. You're twiddling all these parameters and you're not really being able to. There's no, there's no real theory behind it. It's just taking gradients. But it works very.
You could argue it's the, it's the great savior of multivariate calculus.
Oh no, no. We use back prop for. It was just when that came in actually it saved our net talk because bolster machine was taking too long. But as soon as we put back prop in boy would solve it overnight.
So what was the origin of the back prop idea?
Well, I, I remember I was visiting Jeff in Pittsburgh and he told me that Dave Rumelhart had this idea about taking gradients, you know, partial derivatives of the loss with respect to each weight in the network and how you can use a chain rule. And I, and I said that, well, you know that, that's, that's interesting, but it's not as nice as the bolster machine.
But so that's a very mathematical idea for a psychologist to have.
Dave was a really unusually, I would say broad, broadly based psychologist. He was working at the time on language and was trying to understand something about how, you know, with Jay McClellan, something about how words are work together. And, and, and he had the idea that you could use a, a network with constraints in it between different words that could, could settle to some kind of interpretation of what the what, what words went together, noun phrases and things by relaxation and, and, and in, in a sense though, he was beginning to think about dynamics. So I think that that's what led him to this backdrop idea. And, and it was interesting. The PDP books came out in 86, which was when the first paper came out back prop paper in Nature. So they already had that for a few years and I was using a netalk already.
When they started talking about back propagation. What was it? Was it introduced as a kind of applied mathematics idea or was it introduced as a what what in what it was.
By the way, the actual algorithm goes way back. A lot of people had it. In fact, there's a control theory book that goes back to the 40s, I think, where they, they, they just posited that you could solve a class of control theory problems if you did this gradient taking the gradients in a large network. And you know, the problem was that people just couldn't simulate it because the computers couldn't take it. So nobody had actually applied it to anything practical. So Dave along with Jeff were applying it to these networks that we're building. Right. We were simulating all these networks of the bolster machine. So he said, well I'll just take one of the networks and I'll backdrop to it. And boy, was it efficient. Right. Like I said, we used it for Net Talk. It was, it was a summer research project by a graduate student from Princeton, by the way.
But so, so what were the other practical things going on at that time? So there was Net Talk. I remember. I mean what, what were the other things people where people were actually sort of using neural nets and doing stuff that they could demo.
Well, I, I gave you the example earlier about reading. So I had a graduate student from electrical engineering who combined the spectrogram from the auditory signal and then mapped the face with lips into the spectrogram and then combined them and showed that you could improve anybody's speech recognition system with that.
As long as there was also, as I recall, I mean there was the optical character recognition people and I don't really. That was much more of a practical kind of engineering thing.
Yeah, that's right. There were a bunch of engineering. Okay. Engineers particularly like this and applied it to a lot of things. By the way, there are earlier people like Bernie Widger who are using it for simple networks, for equalization in modems and how to shift the frequency bands away from the noise. And so there were a lot of these small engineering applications, non trivial ones, by the way, which were already there and people were using them and this was basically scaling up what they had already done. And by the way, this is something success has a thousand fathers. And we can go back and see how so many people have pieces of the puzzle that worked under special circumstances or for a particular problem. And it wasn't until the whole all those pieces were put together that we saw what it was going to look like. The back prop together with the architecture and then deep networks all of those things had. That was, by the way, it took 20 years to go from the concept they had learning algorithms to something that can solve a really important practical problem. That's because you needed a lot more computing, a lot more data and a lot better algorithms.
But so, I mean, when people were doing optical character recognition, I mean they had been using things, sort of image processing kinds of things and saying when there are these characteristics it must be an A and when these characteristics must be B. But that had turned into a neural net story, certainly by 1980, I think.
Yeah, that was right. So the traditional, in computer vision, the traditional way that you solve a problem is by handcrafting features that were specific for specific objects, that were invariant to say, rotation and projecting an object onto the two dimensional image. And that was very, very slow because it required that you're crafting handcrafting. And also a lot of you also have to worry about whether the feature is present in other objects, right, because they will be. So you have to have a combination of features. And then the question is, you know, how do you do that? Ultimate make the decisions. And it was, in a sense, it's like building a neural network by hand, except that it was, you know, very labor intensive and also took a long time. But the beauty of learning is first of all that that's done automatically. It will figure out what the features are needed at the lowest levels and then it will combine and then be able to solve the invariance problem at a higher level, higher layer. But the beauty, the real beauty of the network learning is that you don't have to be a domain expert, you just have to have lots of data. Number one, number two, the very same architecture with little twiddles in terms of number of layers. The number of units and layer hyper parameters can be applied to any problem. In other words, learning is a universal method and it trades off computing for labor. And if, and now computing is dirt cheap compared to what it was back then. And so learning is a preferred method. And there's still, by the way, I mean, model based solutions are still important and useful for small problems, but they don't scale very well.
Well, I know, I remember in 2010, probably talking to you, I was interested in curating kind of machine vision methods for common kinds of objects. And that was when things like the Viola Jones method for finding faces and so on was there. And that was an image processing based idea. And I was really close to putting a lot of effort into curating 100 different kinds of objects and figuring out what, you know, that was saved by deep learning in 2012 or something.
That's 2012 was the turning point. Yeah. For the famous Imagenet NeurIPS poster presentation by Jeff and a couple of his graduate students. Yeah. By the way, Paul Viola, you know of the Viola Jones face detector. It just detected faces, by the way, which that was a difficult problem. You know, you have an image and where are the faces? Right. And he got his Ph.D. from MIT working in my lab in the, in the 80s. He came to my lab because there's no one else at MIT who was working on neural networks. Huh.
Okay, so. But that was a neural net based thing. I mean, in the end it was.
It was, it was all, it was all a lot of the concepts that went into as entropy. He used entropy and a bunch of other concepts that came from neural networks. No, it's a neural network. It, you know, it clearly had some labor in it in terms of how to, you know, package it into an engineering solution. But no, it was, he was, he was in my lab. I mean, we were talking. Tony Bell was there at the same time. They were in the same room together talking to each other. Tony Bell used entropy for creating an algorithm for independent component analysis. Right. And so Paul went off and created an entropy algorithm for face recognition, face detection.
All right, so let's see, back to the history here. So we're in the mid-1980s, there's back propagation gets invented and there are a few early applications of neural nets. And then my impression was it was, I mean, not that much happened. I mean, the big breakthrough was 2011. 2012. But what was happening in the period from like mid-1980s to that time?
Right. Okay. What happened was that although they look promising, the networks were too small to solve a really difficult, complex problem. Like image recognition databases weren't big enough. And, and so what had to happen really is that there had to be a lot of exploration of network architectures and, and ways of, of eking out a little bit better performance. And I'll give you one example. So every year, pretty much on a clockwork, Jeff would call me up and say, terry, I figured out how the brain works. And he would tell that to other people. We compare stories. But I remember once he called up and he said, you know, we can get 10% better performance out of back propagation. And I said, patent it because, you know, that's going to be valuable someday. And it turned out to be something called dropout. And so here's how it works, which is really interesting because it turns out to be what we see in the brain too. Instead of assuming that the weights are going to be fixed, what you do is on every, every batch of, of inputs that you're training, epochs are called. You, you drop out half the weights in the network, you just cancel it to zero, and then you do back prop. And then in the next epoch you, you wipe out a different random set. And, and that improves the learning, the performance and the learning process by 10%, which is amazing. And, and you know, a lot of reasons for it. We've, interestingly, it turns out that the synapses in your cortex, right, you've got a lot of them. You know, like a million, billion of them are probabilistic. That is to say, on any given input, the, the probability that you have any output at all is only about 10, 20%. So you know, it's again, the brain is filled with these tweaks and little, little improvements on algorithms to make everything much more efficient. And we gradually are discovering that as we explore these network models. And it took like decades to, to, you know, just by brute force, trying out different things. For example, I'll give you another example. This is Yama Kun Convolutional Neural Network. So there were a lot of things that he had to do to make it work, which like one of the things was that he did something called pooling where he had say a lot of feature detectors, say, so vertical edges that were pooled across space. And then he used that as a way for then creating a little bit of spatial invariance for the next layer. Well, that's called a complex cell in the visual cortex. And that's something human weasel discovered back in 59 or 62 was their classic paper. Jay Fizzial. And so what happened was that Jan would systematically discover things like that, like normalization. Like before you pass it on to the next layer, why don't you normalize the activity level so they all have the same average level, and so that you're not saturating when you go up. And again, that's what inhibitory neurons are, negative feedback in, in the cortex. So basically, Jan converged through just trial and error with many different things to the architecture of the visual cortex. Now, the difference is that the neuroscientists knew it was there, but they didn't know why it was there. Right. They didn't know computationally what the complex cell was doing or what the normal, what the inhibitory cells are doing. But Jan actually converged and for computational reasons for those particular features. And now we can go back and then look at the cortex and say, well, what else is there that we don't understand that might also be useful? So there was a back and forth going on back then between the computational neural networks, and that's continued and it's actually blossomed, which is within the last five or 10 years, there's been a tremendous amount of flow of ideas and algorithms back and forth.
So those things like Dropout, for example, that wasn't invented for neurophysiological reasons. That was invented as a, as a kind of algorithm hack.
Oh, exactly. No, no, Jeff is brilliant. I mean, he's a genius at coming up with brilliant hacks that often are ones that nature was there first. Right. Jeff, you know, is.
Okay, so, so you now must have the laundry list of the, of the 25 hacks nature has discovered, of which, you know, the Jeff Hintons and Jan Leun of this world have only discovered 10 or something.
I would say that, that that's less than that. Probably there are many, many others. And I, and I say that because I, I look at the brain and what I see is not one giant network. I see dozens and dozens of different kinds of networks in different brain areas solving different problems very efficiently. And we're just scratching the surface right now with regard to principles that we can extract from the brain. I think the brain will be a source of inspiration for decades to come.
Huh. But, so, okay, so we're now, by the way, when you were doing that talk, was that that was on a Vax type computer, Is that right?
Is that. It was a ridge computer, which was a knockoff of a Vax 7.
I remember the bridge, the, the 32 bit. It was an impressive 32 bit computer.
I had the size of a washing machine. And because I was a, a presidential young investor at this award, I got a 50% discount. So I bought two. I had more computer power in my lab than the entire computer science department, you know, running one of the 780s.
Right, yeah, I remember that. I, I had one of these Ridge. Ridge computers at Institute for Advanced Study. Yes, it was, had that and sun workstations. Those were my two things. The Ridge was a very strange creature. I think it had some risk architecture technology in.
Had. Yeah, very cores I think, you know, was really primitive anyway.
But, but okay, but so that was the, that was the machine and that was okay, but so you're saying in this period from the mid-80s up until 2010 or so, this was a time when, I mean, to me I would hear about these neural net innovations and I was just looking. I collected all these papers over the course of decades about neural nets and I never really did anything with them and it never seemed to be going anywhere. But there was a group of people that continued, I guess to pursue these things. And that group, I mean it was a fairly small group as I, as, I mean it wasn't accumulating a lot more people.
I know all of them because I was one of them.
Yes.
And like I said, the NEUROPS meetings continue to go on for like now 30 years during that period. But the core of the people were part of a group that was funded by the Canadian Institute for Advanced Research C. It's now called CIF A R.
Is that why Jeff Hinton went to Canada?
He went to Toronto? No, he, this was. Maybe it was part of the reason, but no, he went to Toronto. He wanted, you know, the, the, the, the main source of support for computer science at that time was darpa and, and Jeff was a Carnegie Mellon and felt the awkward taking military money, so he went up to Toronto and.
Was there a particular person in Canada who really believed in this kind of direction?
Not really. Okay. No, there were, there were. What was. The CIFAR group was a very interesting group because they would put together a project and so you had to have some interesting project and you, you supply for support and it wasn't enough to actually do a big DARPA project. What it was was to give you some salary relief and maybe have a postdoc. Right. But there's a core of about 10 people and you know, Yann Lakhan, Yoshua Bengio and a bunch of others. But we would, and I was on the scientific advisory board, so I would go to all their meetings. And so I saw this happen over a period of like 10, you know, it actually went on for longer, 20 years. It was originally, it was focused on vision and that's when Jeff was a director and Then when Yoshua took over, it was focused on language. But the beauty of it was that it was a group that would get together regularly. We had summer schools, we had. And Jeff, by the way, spent most of that time working on bolster machines because he really felt that that was where the real breakthrough was going to occur because it had a beautiful theory behind it. It wasn't just a engineering hack. But the beauty was that as the GPUs were a dividing line between the old neural network performance and the new. Because literally within a couple, within a year you could enhance. Because the architecture, as you know, for doing graphics is large matrix operations, right. With vectors and so that you had multiple cores, right. So within a space of a few years you could jump one or two orders of magnitude in the size of the network that you could learn. And that suddenly broke through the threshold for actually solving world class problems in vision, speech recognition, language and so forth. So that was a phase transition and here's why. Imagenet was a huge database of tens of millions of images with tens of thousands of categories, you know, even obscure ones like all kinds of different birds and flowers and so forth, that what most humans probably wouldn't.
Yeah, and lots of shocking ones. I, I looked at it. It had some, it was, it was a, it, it was, it had a certain, you know. Yeah, it was, it was a curiously curated thing. Yes.
But, you know, but it didn't matter what the categories were because, you know, it, it just needed something that would scale everything up and, and progress that was being made in the field was like, with the labor intensive method we discussed earlier was about maybe 1% decrease in the error per year, but the error was huge. Like, it was like performance is like 20% correct.
And this was, this was, you were, I mean, there were these competitions every year.
Yeah. Oh yeah, yeah, yeah, the imagenet competitions. Right. So what happened was that using convolutional neural networks, these two graduate students in Jeff's lab lowered the error in ImageNet by 20%. So it's like 20 years of research overnight as far as computer vision was concerned. And by the way, the way that the community works is all with benchmarks, right. And so if you can do better than the existing algorithms, that you can't compete. So they had to literally phase around is over. Maybe it took a year or, you know, or so until everybody was now using these CNNs.
But. So this was, this was the Alexnet thing, is that right?
Yes, that's right. Alex Khashevsky was one of his Graduates.
And so, I mean, I think there's a story that's told that it was just like, well, they left it training for a month and they forgot about it and then came back and found out that it had been successful. Is that story true or is that an apocryphal story?
I, you know, I, I'll have to ask Jeff that. I don't, I don't. I, you know, I didn't. I've never heard that story, but it's, it's, it's. I think it sounds like it could be apocryphal. By the way, by the way, as you're talking about apocryphal stories, I once heard a story that the first diaper graph from MIT was to build a robot that played ping pong. And when they got it, they discovered they forgot to ask money for to. To write a vision program. And I have always thought that was apocryphal, because at the very same meeting at Dartmouth where I met Mark Minsky, I asked him, I said, I heard the story, and it's like, to find out whether or not it's true, he says, you're wrong. We did not give it to a graduate student as a project. Summer project. We gave it to undergraduates.
Hmm. Okay. To solve the vision part of it.
Yeah, that's right. So that's how. And I actually found an AI memo to that effect.
By the way, had DARPA supported a lot of neural net stuff? I don't remember DARPA being deeply involved in neural nets. Seems like that kind of thing.
No, no, no. DARPA was funding Marvin Minsky to write a computer program that could play ping pong. Right.
But not using the AI lab.
And it had nothing to do with their own network.
Right.
They were going to build a robot and they were going to write programs that did vision, and they were. They were going to assign a graduate student as a summer project to write a computer program, the vision program. That, that was how naive they were about the complexity of the problem. And even, you know, after that, that was. That was probably in like 50. Early, like 60. The 56, probably 1960 or so. And, you know, and just think about it, right? Even in the 80s, people were still struggling in computer vision.
Sure, right. But I mean, that was a different tradition. That was the symbolic tradition of AI, which, you know, I think. Well, this is a long discussion, but probably not for. But, but so, so coming back to. Okay, so GPUs. When. When did GPUs come on the scene? I'm Trying to remember that must have.
Been, let's see, 2000, around 2000 or so, maybe a little bit later.
But so, so then there was a 10 year period before there was sort of breakthrough work done in neural nets based on GPUs.
Oh, oh, no, no, no, no. I'm sorry, I miss, I was thinking to about 2010, I would have, I would have guessed they picked it up very early. I remember Yann Lecun at one of the CIFAR meetings saying that he has a graduate student who is hacking and that it looked very promising. And, and so yeah, as soon as that they got that up and running, everybody in neural networks, you know, realized that this, this was like, you know, exactly the right architecture for the problems that we have. There's a CUDA program, the programming language that you could use, so you didn't have to microcode it. It was, it was very efficient.
Right, but so I mean the sequence that, you know, there was the imagenet breakthrough.
And then by the way, actually it wasn't, that was the big public breakthrough, but there was a breakthrough earlier at Microsoft. Jeff sent one of his graduate students there to help them with speech recognition and apparently it had, it really was working much more effectively than, than their program which was based on Markoff hidden Markov models. You know, there was a bound Welch algorithm for.
So when was Leanne Leun's Lynette stuff that was, was that.
I, that was actually much earlier. That's when he was at Bell Labs. At home though, there was a group there, Larry Jackal and a bunch of other Sarasoya and the, let's see, who was the other one? I mean there was just a really a powerful group of engineers there that were solving these practical problems and Jan was one of them. Isabel Gill was there at the time.
So this was handwritten digit recognition. That was the, was that the, that's the.
They had this database from mnist, it was from some post office I think in Rochester, New York. And it was, you know, they had thousands and thousands of zip codes which had been handkey. I mean we're talking like 700 pixels, right?
But the way, the way that letters were being routed at that point, people were, were keying in, they saw this, a human would see the zip code and then they would put in, in whatever, you know, this invisible ink or something. They would actually put the, the, the, the data on what, the actual zip code?
Yeah, they had to. Because if a human being was the only image processing system that could do that at the time.
Right.
And And I think, I think that there were competing systems which weren't as accurate.
But, but so how had those guys at Bell Labs, how had they gotten into neural nets?
And what was, you know, it's interesting, they were the, I think Larry Jackal for example was actually, he worked on chips, he built chips. Chips. And so they, there was a whole group of people at Caltech and at home, Dell and a bunch of other places that realized that we could speed things up even further by special purpose hardware. And so they, they were designing chips that could do these things.
Was that a kind of Carver Mead related enterprise or was that not was his effort yet a separate branch?
Which branch?
But the COVID Meadow.
Oh, Carver Mead. No, no, Carver, the, the neuromorphs. Actually I'm as one of the founders of the Telluride workshop, Neuromorphic engineering workshop. So I know everybody and, and, and they were, Caltech was one of the hotbeds for the neural network people. In fact, the Dimitri Salches is there who's trying to build the first generation of optical neural network. And now it's making a comeback because it's actually a very practical solution to a lot of these.
When was that?
That was in the 80s probably, you know, about the time 87 was the first Neurips meeting. And then it was probably, yeah, so.
It was after I left Caltech in 83. So I, I, I thought I, you missed that.
Oh, and the CNS program. So they had a computation of neural systems which was jointly with biology and they had a lot of people building things, neuro chips and things. But they, you know, Caltech was, was way ahead of a lot of other engineering institutions which were still back in the dark ages. You know, they didn't, they didn't understand what, how the importance of learning. They didn't understand multi layer networks and.
But so the, this group at Bell Labs, that Holmdel, where had they come from? I mean, I think Jan Lecombe was a physicist originally. Is that not correct?
He, he, yeah, he comes from the, I know, I think more of an engineering background. Larry Jackal was a physicist.
Yeah.
So I think, well, I might be wrong, but I thought Jan was coming from computer science. But, but he had a version of back prop that he came up with on his own for his thesis. And I remember I met him in Le Zouche, which is a physics school, summer school. And, and he, it was, it was wonderful because you know, here's this poor, that poor guy. He was, came from a good school but isolated. Right. There was nobody around him at all, to talk to at all. And he came to this workshop and he, he was so happy to talk to somebody.
And what was the workshop about? I mean there was. Because lazy is a very physicist organized.
Well, no, no, it was, it was, Neural networks are pretty hot at the time. I mean this was, I think it was already past the Hopfield network was already in vogue and there was physicists were, I, I have to say that the I, it was, it was, they, physicists, I think had the right way of thinking about the problem. But, but I think what, here's what was missing in physics was that you solve a problem in physics by writing down the Hamiltonian and solving the equations, right?
I mean, magnificence, I'm not sure that's, I, I, I don't consider that a universe.
I, I, I'm not saying that that's all the problems, but I'm saying that, you know, you have, you have a Hamiltonian for the hydrogen atom, right? And you have to, quantum mechanics, you, you solve it with quantum mechanics. Okay, but, but no one in physics, at least not, not mainline physics thought about was what if you are able to change the Hamiltonian right now, what if you have power over physics, power over the Hamiltonian, what could you build, right? So that's a different mindset because now you have, that's what learning is all about, is how, how you can, you can implant the complexity of the world into your brain through learning. And that's like letting nature kind of adapt to the environment.
That's an interesting interpretation there. I mean, obviously the physicists around the early 1980s were all interested in spin glasses where you would have this whole pattern of different weights effectively in the spin glass. So your claim is that basically they never really imagined the idea of having, I mean people, like even the early simulated annealing stuff was about finding energy minima in spin glasses, right?
So the spin glasses people would study random networks, you know, so you could prove analytic solutions to what the landscapes look like. But yeah, and actually Hopfield, one of his contributions was to show that how do you put the local minima in? It turns out you do it with Hebbian learning. You take the vector, right? And, and then you take the cross product and then put those into the.
Weights and that, that was the original Hopfield network idea, if I remember.
Yeah, that's it, that's how Hopfield created the local minima. The, the, the tractor states. Right.
That's a, it's an interesting perspective that the, that the, the thing missed by the physicist was that you could change the weight, so to speak.
Right, that's right. At your, at your will.
Right, but so, okay, so then we get to 2010 and basically GPUs come in and one starts to be able to do all sorts of. And basically, then the basic. You're basically saying that given GPUs, you could make the old neural net ideas work. Is that a fair.
You could jump. Okay, so Moore's law doubles every 18 months. Right.
So how stopped a while ago. It was supposed to, it didn't work. It hasn't been working for a while.
Okay, but it's it's some, something like, you know.
No, that, that was the original Moore's Law.
Let's just say a year. Okay, let's just say a year. It doubles every year. So to get to 100, you would have to do like two to the six.
You know, give or take.
Yeah, six, two to the sixth. So you. Well, I guess that's like six years, but 10, you know, with the 18 month wars law. So it's like a decade. It's like jumping a decade into the future. Right. And, and, and that is a, a major kind of leap into a larger scale. And, and by the way, we don't really quite understand what determines by the way. The speech recognition, vision and language all made this transition within about five years, you know, less than 10 years, in other words.
Well, it happened at different times, right?
I mean, it happened at different times because people were working on the problem at different times and, and they didn't all have the same access to the, the tools that the Google had and so forth. So for the transformers. But the, the reality, I think is that it may be something similar to what happened in nature. So very, very few things scale up. Very few algorithms scale up the way that brains have and also these large language models and deep neural neural networks. So if you look at the size of primate brains, which are on average per body weight, much bigger, the cortex has expanded even faster in terms of the amount of neurons and the connectivity within the cortex. In fact, in the case of the human brain, it's gotten to get it inside of a reasonable volume, you've got to have these convolutions, right? You kind of squeezing it in, right. And if you look at the cortex, although there are differences from area to area that uses the same six layers and inputs and outputs. And so there's this is, you know, is it more. It is better. Something was good about that architecture, which wasn't the case with a Lot of other parts of the brain like the, the amygdala or the hippocampus. Hippocampus actually just stayed the same. It didn't expand, even though it's really important. But it, it didn't. But the, as, as you add more computing into the cortex, it, at some point you start getting capabilities that weren't there before, like language processing. Right. And so it, it, it's, it's the, the. In computer science, I've learned that you have to look at how each algorithm scales and you know, the worst is NP complete, in which case it's exponential. But you know, even polynomial kills you at a certain point because you know it's, it's going to eventually use up all the resources. Beauty of these networks, they scale with as N. Right. The number of weights. It's linear.
Well, not yet. I mean the learning part of it with back propagation is more like an N squared story. I mean the actual backpropagation process.
No, no, I'm talking about, I'm running it real time. Just the inference part.
Yeah, okay.
And that's all that matters really, you know, in terms of using it to solve problems. You know, the offline learning and everything is, you're right. It takes an enormous amount of energy and so forth and eventually we'll figure out how to do it online. But not, not yet. Like they call it lifelong learning. But the, there is sorry here because if you now build the special purpose hardware, nature built the hardware so that it could take an order N problem with N weights and actually build all the weights so they're physical. It becomes an order one problem. Right. You could run it in real time. You don't have to simulate it. It just runs. And, and that is very, very, very few algorithms are, are can scale that way. And now there's a huge, huge push for building special purpose machine learning, digital machine learning and neuromorphic engineering. It takes it one step further and takes it down to building special purpose networks which are based on the same biophysics that your neurons are based on.
But so if you look at the biological in the actual history of neural networks, there's 2012, big progress in image recognition, then there's, I don't know when the speech to text that must have been. When was that? That was like four or five years ago now. And now large language models and so on. And I'm curious whether that recapitulates anything that you see in actual history of biological evolution. Oh, well, I mean, trilobites.
No, no, you're absolutely Right. You're absolutely right that the transition for transformers came about 10 years later. Not quite. I mean it was about three years ago, so it was less than 10 years. But it, but it required a much bigger network than imagenet, Alex and I, and, and, and speech recognition. So you're, you're right. It wasn't all at the same time. You're right, but I was exaggerating. It, it occurred over a time period where things were expanding very rapidly in terms of the sizes of the network, you know, primarily at places like Google, Microsoft. But, but I, you know, maybe, you know, for all we know, it's going to continue. Right? It's not, it's not going to stop. It's, it's because, you know, you just add more.
Right.
Of these GPUs, or now they have the TPUs and Google and other special purpose chips that are going to make it even faster and cheaper and eventually. Okay, I did this calculation. So how many, okay, 100, 175 billion weights in, in GPT3. Right. So that, that's how many, how much cortex does that correspond to? It corresponds to the number of synapses under a square centimeter of cortex and you have about a thousand square centimeters in your brain of cortex.
So wait a minute, so for each neuron, so they're like 100 billion neurons in the brain and each neuron has what, 10,000 connections or something?
Yeah, roughly 10 to the 14 total. Yeah, there's about, there's 10 to the.
That will be only a thousand.
10 to the 11th neurons and, and well, okay, 10 to 10 to 11th neurons and about 10 to the fourth connections each. So that's like 10 to 15. It's not quite that high, but between 10 to 14. 10 to 15. But. So that's like a million billion synapses.
Right.
And, and so a couple, the 10th of the. Let's now there's up to a trillion. I think the networks have a trillion of these weights in them. So you know that that's 10 to the 12th. So there's still like a thousand factor of a thousand away from just the raw storage capacity. Although a synapse is much more sophisticated than a single weight. I was telling you the complexity is, it's more like a dynamical system. So you have to add another factor of 10 or something.
On the other hand, the, the clock speed is an awful lot lower in a brain than it is in a.
In a. Ah, okay. Now is that a feature or is that a bug or a feature? Okay, so here is Again, digital computers go really fast and they have buffers and they can do things faster than we can. But they, but if you have a network that solves problems in real time, you don't need those buffers or very little. You just have a little bit of working memory. And the idea though is that the time scale for processing in the brain matches the timescale of the world. You don't have to go faster than that.
Well, that's defined, the timescale of the world is defined by the timescales we think about. I mean, there are plenty of things in physics that happen on much shorter time scales. The ones humans are aware of. I mean, you know, the speed of light is very fast.
Oh, you know, no, no, actually, you know, things are happening at synapses, at the microsecond level. I'm talking about behavior of the whole brain, right. In other words, 100 milliseconds to recognize something, there's a lot of things flying around inside, you know, to do that. So. Yeah, but I guess, but even that is now millisecond we're talking about, you know, if the synapses, time constants of a millisecond, the fast ones, there's some that are even much slower. Okay, well, but you know, the world works on a millisecond timescale. You know, you, you catch a ball and, and you have to judge things with a millisecond timescale.
And, and I think this is a self fulfilling loop. I mean it works on millisecond timescales because that's the timescale of our thought processes.
Yeah. You know, I, I, all I'm saying, by the way, okay, if it is possible for nature to go much faster, but at great expense. For example, the giant axon in the squid, it's a millimeter in diameter. Just to show you the reason is that the larger the diameter, the faster you can get an action potential. And the squid has an escape response. So when it is in danger, what happens is it suddenly activates that nerve and then it shoots off as a jet of water that is now released to make the thing just scoot off faster than other fish or you know, whatever is whales or whatever are going to, can go in, in that, in a random direction. So, so with that, with that. So, and then there are other things. For example, your ability to detect the location of a sound. And, and it turns out that you can do that because nature has, can do comparisons of time differences between the two ears in the microsecond range. Okay, so if, if nature needs to do that. Yeah, it can, it can, it can engineer with the existing circuits, but it only does that when it's really pushed to the wall, where having that can save your life or it can give you some capability that is important for your survival. So that's, you know, and the, the. I, I think that it's a beautiful thing that the thing works in real time and that it, it's, it's engineered to do that. And by the way, again, you, you can do these fancy things, but at the great expense in terms of volume, the amount of material you need, and also the energy you have to put in. So nature has taken the low road and figured out what's the least amount of energy I could use. Take, you know, if, if it could solve the problem in millisecond, then let's go with a millisecond. Right. Or, you know, muscles, even, Even slower.
Right. I mean, you could summarize that by. Who needs computers when you have brains?
I didn't say that, but I agree.
But, but, so if you look at the. I mean, for example, let's take ChatGPT. What's the next stage? I mean, so if you're right, that, that corresponds to, you know, what we have right now with a trillion weights is, you know, 1 square centimeter of cortex. How big is the total area of the cortex?
About a thousand square centimeters. Okay, so about the size of a dinner napkin.
Okay. And does any other. Other creatures have bigger ones, but they also have much larger body weights?
It turns out that the, the, the b. The brain scales as the three halves power of the body weight. And that's because that's the surface area of the body, which is the receptors. And, and everything scales according to that. I see receptors.
So the whale. So a whale has a lot of nerves coming in from all of its different flippers and so on. That has to find a place in the.
But that's the first approximation. And like I say, there are exceptions. Like the primates are on a. They, they, they, they, they can still, they still scale with body weight, but on a, On a slightly different line in, in that space. Okay, but is, but so the. Go back to your question, which is, you know, what's the. What, what, what, what. Where are we headed? And what. What's missing? Okay, so first of all, I made a list. I have a, if anybody's interested, an archive paper that's coming out next month in neural computation on. It's called Large Language Models and the Reverse Turing Test. And if you're curious about the reverse Turing Test, take a look. Wait, wait. Steven passed the reverse Turing Test.
Wait a minute. What is the reverse Turing Test?
Well, it turns out different people come away thinking different things. Some people come away saying, this is amazing. I can't believe it. Other people say, this is. It makes mistakes. This is terrible. I mean, it's. You know, it's not like humans that don't make mistakes. Right.
Right, right.
And it. And what I do is actually have about three different dialogues with three different people who range in terms of their conclusions about whether they're talking to something that is incredibly intelligent or something that's incredibly dumb. And, you know, there's a range in between. It's a huge debate out there. I mean, it's going on and on. Now it's reached the public, and people are going off in different directions. But the reverse Turing Test is the following. So I came to the conclusion that what you get out of it is what you put in. So if somebody asks intelligent questions, you get intelligent answers, but if you ask stupid questions, you get stupid answers. And the reason is that the GPT3 has access to the world's database, which includes the world's smartest people and the world's stupidest people. It takes on whatever Persona.
You're saying. If your prompt. If your prompt is characteristic of one type of person, then you're going to be sampling the part of its training set.
It will look through its training set and say, is there a person like this out there? Yeah, there are a lot of people out there saying stupid things. So that must be what I'm expected to do is to be a stupid person.
Well, alternatively, it's kind of like this. What is it? You know, these. These kind of echo chamber type things you're saying? Basically, it's. It's the. It's the perfect.
It's reflecting. It's mirroring your. The quality of your prompt. It mirrors the answer back with the same level of complexity.
Or given that assuming that the ocean of what's out there on the web and in books and so on, that it's been able to sample that assuming that you are somewhere in that ocean, the prompt you produce will, you know, the fish that are around that part of the ocean will be what you'll get back from the prompt you give.
Exactly.
Somewhere that isn't even in the ocean, then it's a different story.
Exactly. So, you know, that's a. That's wonderful. That's a wonderful metaphor. And so that's the idea is that you have this ocean was filled with all kinds of different creatures, really smart ones, really stupid ones, and everything in between. And, and, and, and GPT has to, when they agit a query, they got to figure out where in the ocean am I going to go? Where are there, where are there discussions like this, you know, that I've, I've experienced before. And if it's a really interesting question, deep question where a lot of people have deep thoughts, it will go there and mine that part of the ocean. Right. As for responses, right, but so I'm.
Curious, I mean one of the things, you know, it is, apart from its outer loop, it is a feed forward type network. And so there are severe limitations on what it can expect to compute. And I guess one question would be if you talk about learning as a universal, you know, meta algorithm, so to speak, one of the things that, you know, we're not particularly good at it, but we do manage to learn things which are non trivial kind of looping procedures, so to speak. What, what I mean, is it obvious what the, I mean because, because the learning methods used for, for you know, GPT3 and so on are as we've just been discussing, they're, they're very classical learning methods in the, in the, in the history of neural networks works. I mean, you know, is that how, how does one learn loops? How does one learn things that are more complicated than kind of feed forward kinds of things?
Okay, so, okay, so, so the, the, the, the whatever. Okay, you brought up the issue of what is it that the transformer is learning. It's basically taught to predict the next word in a sense, right? That's it. And once you've created the internal representations which take advantage by the way of anything possible, including the semantics, the syntax, all of the relationships in terms of higher order statistics that might be relevant for different. The text that it's encountered to solve that problem, it can now, now be used to generate as a generative network. It can generate new texts. That is really original by the way. It's not, it's not copying people think oh, it's cutting and pasting from all those things. No, it would never, it would never be able to do that. Never, never in a million years. These networks generalize from the examples it's given. It's given examples that it may never have gotten before anywhere in the world.
Right? But the fact is, you know, if you, if you give it a big piece of a well known document, it will continue verbatim because it will have enough Information that's encoded, I mean basically will have successfully encoded in its weights, you know, that precise statistical continuation.
Well, I would be surprised if it was a perfect verbatim. But yes, it would be able to replicate something that was well established, well known. But if, I'm pretty sure if you just take a random sentence from a random blog in, in the Internet.
No, I think you're right. I think what happens is if you're, if you're out in the, in the corners of the distribution where there's very little support out in the corner of the distribution and there's only one thing out there, it's going to be something. It's going to look very much like that thing. But if you're, if you're in a place where there's a quite high population of things, it's going to do all kinds of different stuff.
That's my intuition too. Yeah. And it does have to compress, it has to compress all that data that's in the high density regions. And by the way, it's not good at extrapolating. That's one of the problems is how do you go beyond the text that you have and look at out of distribution. It's called, it's a very hot topic in machine learning right now.
Right, but I mean in fact the number of weights is very comparable to the number of tokens of trans training data. I mean that was.
Yeah, well that's one of the, that's another one of the things that we were told back in the 80s that we, we had were underpowered, that they would over, you know, that the network would memorize basically which, which didn't happen actually. Even though we had relatively small databases, we had regularization techniques like weight decay which helped but, but actually now people actually have found that there are rules about how much data you need, but it's much, much less than the complexity algorithm. The complexity equations that are, came from statistics which would have predicted much, much larger databases which are based on wrong assumptions though about the high dimensional spaces.
Well, I mean, my impression is, you know, if you're trying to learn some, you know, wiggly mathematical function, you know, you might actually need quite a lot of examples, it turns out. I suspect that there's more regularity in language than people knew and that's why it's possible to learn, you know, possible to represent that with a.
That's exactly right. And that's what these networks are designed to do is to pick up regularities that may not be apparent but allow you to compress and maybe are able to interpolate within the complex function that you have. And, and, and by the way, the, it, it, it, it with the reason why deep learning and, and when you have many layers is useful is that the way that you do that is by starting out with trying to understand what's, what's happening locally, you know, in terms of small amounts of data coming in and then gradually build up larger, more abstract representations as you go up. So that becomes easier than to do, to generalize as you go at higher, higher levels.
Right, but if, if you stick a probe inside a deep learning network and you say what's going on in here? You know, you'll get all kinds of complicated stuff coming out. If you stick a probe inside a brain and say what's going on in here? You get all kinds of complicated stuff coming out.
That's right.
You know, in either case, do you think that there will be kind of a, or there can be sort of a human understandable theory of what you're seeing?
It's too early to tell, but I know that some really smart mathematicians are working on that problem as we speak. And we already have an indication that there is going to be a real insight coming from brain recordings. So you know, we just went through the numbers the brain has, you know, the billions and billions of synapses and so forth. And so you might think that this high dimensional space is somehow, you need all of it at the same time, you know, to be able to process. But, but now we have been able to sample from, save a hundred thousand routinely now at the same time from multiple brain areas. What we've discovered is that if you have an animal doing a task, relatively simple task in terms of discrimination or judging, making a decision about whether to go right or left, but nonetheless, you know, for animals, right, that's important. And they, they learn how to do that. But under all of the cases that we have studied so far, it looks as if the part of the space of, of, of all the activities and all the neurons that are actually carrying the information that you need is a very low dimensional subspace out of the all, you know, billions and billions of dimensions out there. On any given trial, you're only using typically four to six dimensions. So that means that embedded in this very high dimensional space are these little tunnels that are carrying your behavior along. And a lot of them are very well greased because you use them over and over again, right? Those are the motor activities that you learn, for example, when you play tennis. That's why you get very, very accurate because you can be very precise with that tunnel. But the actual geometry of that space is very difficult to understand. And I'm working with a Fields medalist, actually Stanislav Smirnoff, who's actually gotten interested in that problem. And he is a very, very smart. I don't know if you know him. Very, very.
Yeah, I do know him. I do know him. He's been, he's. One of his other activities is studying pigmentation patterns on various kinds of.
Yes, and he was here just a few weeks ago, gave us a beautiful talk on that. It was a cover article in Nature. It was how the lizard got its spots.
Right, right, right. It was using cellular automata and things to study these. It's been on my list to drop in, in Zurich and see the Bavarian without the actual creatures, so to speak.
Yes, yes, no, it's fascinating. The whole story is just amazing. But that's biology.
Right, but so you know, what is not obvious is when you say it's a low dimensional space. By even describing that and talking about it in terms of variables and dimensions and so on, you're kind of assuming a certain kind of mathematical structure which is far from obvious to me at least really exists there. I mean, in other words, if I started talking about Turing machines and saying what they do and talking about what dimensional spaces they live on, it will be a very bizarre way of talking about those essentially computational processes. So it's, it's an interesting claim that you can think about things in terms of sort of geometrical continuous mathematics like that.
Well, you know, it's, I guess it was, it was, it was a great surprise for me and for many, many others when people started reporting that Dave Tank did these experiments in Princeton. What he did was he actually, it took enormous amount of computer time to do this. What you have to do is you have to follow the whole, you record from, I don't know, a hundred thousand neurons. And what you do is you are now reducing the dimensionality of that very high dimensional space. And you're mapping as this population shifts in the actual brain. You're following it in this transforms low dimension, transform space, right as, as you go through. And you've got to do like, you have to follow it and you have to do incremental PCAs and so forth. It's, it's very crude. But now people are finding many, many, much, much better algorithms to do that. And so, so now, so it's all about. One of the, the big ironies of, of machine learning is that it was inspired originally neurips by the brain. Now all those algorithms are being used to analyze data from the brain. Yes. And you know, and so that's why we're using machine learning to look at the geometry of these spaces and very difficult to have any intuition about what's happening in these spaces. You really have to use these tools and machine learning to do that.
Right? No, I mean, look, I, I looked at the dimension reduction of how GPT3 completes sentences, which is, you know, so the question is, is there sort of a, A, a semantic law of motion that you get from kind of how it is completing things in language, how it is moving through that, through that space?
Right.
It's a bit of a mess so far but you know, it doesn't mean there isn't, there isn't something to be found there.
It's a mess if you expect it to be something that you can imagine. See, we're really very unimaginative creatures because we live in a three dimensional space with time. And now the brain is living in this billions of dimensions and the properties of high dimensional spaces is very, very counterintuitive. I, a lot of these theorems don't make any sense.
Well, in fact, the 2012 breakthrough in a sense, as I understand it, it was a story of, you know, people thought maybe, maybe this is correct, maybe not that, you know, gradient descent and so on. Finding that minimum loss configuration would be something where, you know, if somebody said, well, let's go to a higher dimensional space, most people like me would have said it's going to be more difficult in a higher dimensional space. But that turned out not to be true.
Just the opposite. The bigger the space, the easier it is to find a solution. And now what's happening is very interesting because it's possible. Once you have that solution, you take the output from this very large network and train up a smaller network to give you the same output. And so you can distill it. It's called distillation. But you need the big network to find it in the first place because if you start from the small network, you'll never make it. You'll have you, you don't have enough degrees of freedom.
So that's, you're also getting training data and so on.
And yeah, it's, it's the nature of how you need the extra dimensions in order to explore the space efficiently. The other thing, by the way, the big difference was, you know, between the people back in the 80s who were telling us that it'll never work because you get these local minima, your gradient descent, non convex spaces is just, this is a non starter and we're told that we are, you know, we're going to be overfitting, we don't have enough data. Right. And that's coming from classical statistics where you have small models, you're trying to find the optimal set of parameters for that model. A single set of parameters. Right. And what, and the assumption, that assumption, if you just say, okay, I don't care how many parameters or what their values are, I just want to solve the problem, right. That, that gives you, throws you into a much higher dimension. And, and, and if you repeat the same learning process over and over, you get different networks which have roughly the same performance, but they're all different in terms of the weights between all the units. Right. They're, they're, you can't map one into the other, which means that there's a degeneracy in the number of solutions. There are many, many, many possible solutions depending on what, where you start in that weight space. And that's wonderful because it means that the bigger the space, the faster you're going to get to one of those locations.
Right, right. I mean. Right. But you know, the classical statistics, the failure of people to estimate, you know, all these questions about you've got this problem you're trying to solve, maybe it's playing some game, maybe it's generating language. The thing that was always confusing to me is one didn't know how hard those problems were. Like, I remember, you know, Jerry Tesaro, you remember him? You, you. Oh yes. He worked in the center that I had at University of Orlando.
He was at the institute, I think with you, wasn't he?
Yes, yes, he was. Yeah. He was a postdoc with me and he came. I never understood the work he was doing. He worked with you, which was great, but I never understood what he was doing.
Well, well, he, he did something really amazing. Okay, I'll tell you very simple. We wanted to pick a game that, for which we could show that this approach could outdo humans. Right? Because that's how you get people's attention. So we pick backgammon. Backgammon is interesting because it has a very high fan out ratio and it's probabilistic because you throw the dice. And so we got amassed a large data set from books, from games that are published. And then we brought in people who were masters level, not grandmaster, but people who were good these days, played many, many games. We accumulated a big database and we trained Up a fedora network. And again we didn't have a large network, so it had to be only one layer, I think, or maybe two layers. But it played surprisingly not a high level, but it played at a good amateur level. Right. It was playing the game and you know, it was, we didn't. It was learned. It learned how to play it on its own. Now here's what, what happened now that's when he was at the Institute now but now he went. His first job was at again at IBM, Thomas J. Watson. And it was there that he had. He. He made a brilliant shift away from just training it to take. We trained it to take the board and to predict the value of, of that board, you know, for. So that you knew what move is going to be the best move. So what he did was he had the, the network play itself over and over again. And of course at the beginning it's just random and eventually one is able to win the game by, by you know, just random chance. But then every time it one side wins the all the weights are updated in that value function and it keeps going back and forth. And he used temporal difference learning, which is a reinforcement learning algorithm for updating the weights and back prop for the value function. But ultimately the whole thing got better and better and better and it got to the point where there was nobody there that could beat it. So he had to get some grand master from New York, Roberti I think is his name, who wrote books on, you know, the best, the best moves, you know the. For. For backgammon players. It came in and, and it, he said that. Well, first of all he said this is better than any program I've ever played before. It's really playing at a very high level. You know, it won a significant number of the games not I don't know what the maybe 34% or something. And he said but it made moves that I've never seen before anybody make. And he took them back to New York and played them out and turned out that they were better than the human move. Right? So it was creative. It actually was able to discover new moves that, that no human had ever thought of before. But then okay, the, because the computers were getting faster and playing more and more games. A million games, you know, 2 million games. He came back and he said this now has reached not just the Grandmaster level, but it's probably a world champion. And if it played the world champion it might even win. So that was an achievement and it was overshadowed when go using exactly the same self play and reinforcement learning, but much more computer power, was able to beat the world champion and go. That got the world's attention. I think Jerry Deter should deserve much more credit for doing the equivalent thing with the computing power that was available to him at the time. He basically figured out how to, how to, how to make progress on games. And, and now it's, it's really played out really well in terms of the performance as the networks get bigger, more computing and so forth that you can play more complex games. In fact, it got to the point where DeepMind was able to, it actually was able to bootstrap itself up without even knowing the rules of the game. I think it was able to reach a level where it beat alpha zero. No, it beat alpha go like 100 games to zero. It was called alpha zero actually.
Right, yeah, no, I mean for me it was an interesting that the, the. I chalk up my complete failure to understand the significance of Jerry Tesaro's work back in 87ish time type, time frame.
Right.
And you know, part of the problem that I had was I had no idea how difficult playing backgammon is as a kind of computational problem. And so somebody could tell me. You know, I remember when I first saw, I happened to see the sort of first public demo of the DeepMind, you know, playing eight bit video games or something, and it was a sort of fairly small event. And somebody turns to me and says, well, you know, how impressed are you at this? And I said, I have no idea because I don't know how hard this problem is. Which is the same statement you're saying the statistics folk were telling you if you do all this stuff you're going to overfit and so on. I think the difficulty is for these human like tasks, it's very hard, at least so far as I know, to really have an estimate of how difficult is this. We have measures of difficulty like, you know, we've got for combinatorial problems. We have, you know, computational complexity theory and NP problems and NP completeness, all those kinds of things. But for these more human like problems, it's not obvious what the kind of, what the, what the human version of computational complexity theory is.
Well, in some way, if you have a network that solves the problem, you know, that the complexity can't be more than that. Whatever the resources are that you had to use.
The question is what does solve the problem mean? Because for example, with ChatGPT, if the problem is write a high school level essay, then you know, which is a, again, a very Fuzzy, you know, threshold of what does it mean to solve the problem?
Okay, there you go. So in other words, this is always a problem in AI is that, you know, as soon as you solve a problem that couldn't be solved before, people say, well, you know, that that's not AI, that's not general intelligence.
Eventually they'll stop saying that. We've been through so many iterations of the, that in the last 50 years. I have to believe that's eventually.
Right. You know, but your point is, what's the metric? And how do we know how complex problems are? I think we're beginning to understand them, but it's, it's not. The metric we're going to use is not. It might be having to do with information theory or some sort. But I, I think, I don't think.
So because, I mean, this is presumably the mistake in thinking about language. People were thinking about, you know, Shannon style, you know, Markov chain type things. And, and then it's, that's not really a good representation of, you know, of all possible sentences one can say in combinations of letters, the ones that are meaningful are available.
Okay, so I think you're right. And here's, here's the insight. And I think that, okay, so, so if you look at Go, you know, it played itself 100 million times, probably, who knows? But if you just add up all of the, the, the, the moves, all the game positions, and in 100 million games, it's like maybe, you know, 100 billion, I don't know. But, you know, the total number of possible board positions is some Google plus, right? 10 to the 10th to the 10th or something. It's, it's so giant that the number of training examples it was given was infinitesimal. It was measure zero on that whole set of all possible. But the point is that under all circumstances that humans play, those random positions never occur, would never have ever occurred. And so somehow in this very small subset, there are regularities that represent the complexity of the problem, the true complexity of the problem. Not all the possible solutions possible. Game positions. Intuitively, same thing if I look at all possible images, right? So if you have a megapixel and each one has ten gray levels, right, that's ten to the seventh possible images.
Yeah, right. I mean.
Ten to the million.
Yeah, that's right.
Ten to a million, roughly.
Yeah, right.
And, and so that's a huge number. And the fact is that the ones that actually occur in the real world is this is a very small fraction of, of all of those. Right?
Right, right. The actual physics that we perceive is a tiny fraction of all of the, you know, the images, the physics, etc. They're all tiny slices of what could be.
That's right.
And, and characterizing that, you know, those slices and, and seeing what, what it takes, you know, characterizing kind of what, what the, what the contours of what, what is actually possible are.
That's right. And that's geometry. That's geometry. In other words, the geometry in the images of all the images from the real world and figuring out what's common with all of those images is what learning does for you. It pulls out something that is essential about real images.
Right.
Physics from physics generated, you know, the.
Very assumption that there is geometry assumes things about sort of space and all those kinds of things. We should probably, I'm being reminded we've been, we've been yakking on. We almost made it through the whole history from the last hundred years or something of New Earth, but you haven't.
Asked me about the future.
Well, I think that may be. Maybe we have to do that a different time.
We will. Okay.
But let's see. I think there might be people here. Other questions. Yeah, questions. We have some questions here. Your question about Dennis Gabor's 1959. What's that?
Is. Okay, what was it? 59 paper. Was it or what was it?
Yeah, yeah, yeah. Electronic inventions and their impact on civilization. First proposal of random circuits with weight tuning to learn a black box function. So I think didn't Gabor had had a bunch of stuff that came from sort of the optical holography type tradition.
Yeah, holography. That's a very interesting paper and also is very, very relevant because remember I told you that the original Perceptron, Frank Rosenblatt's Perceptron, could only do linearly several functions? Well, it turns out that he actually, he knew that. And so what he did was he put another layer of weights in between the input and the perceptron units and made them all random, so random projections. And now you know what he did, what he did there, which allowed him to solve more complex problems. But, you know, what he did was what we now call compressed sensing. In other words, you can map the original space into a big, you know, a different space in which it's possible now because the features you've created are kind of random features. And now those random features can be weighted in a way to solve a more complex problem. And there's a beautiful theory of this. It's called the Johnson Linden Strauss theorem, very beautiful. Which underlies all of these compressed sensing algorithms. And interestingly, there's evidence that it's used in the brain because there are parts of the brain which have that character in terms of random projections.
Okay. So the thing grows randomly, and that randomness is, you know, seeing through. Is kind of seeing through some random glasses, so to speak.
The thing that I don't think Gabor had was the idea of learning algorithms. In other words, the closest thing that comes to mind is something called reservoir computing, where you start with a random network that has random activity patterns. And what you do is you train the. The output layer to do some function, you know, so you suppose you want to replicate some function. You can do that, and it works. It's amazing how well it works.
It's.
It's. It's very.
So what would be your expectation if. If I walked around wearing randomizing glasses where every pixel. Where, where, you know, every pixel of. Out of input is. Is randomized before it gets into my eyes, Would I be able to untangle that?
Well, your visual system makes the assumption that it's not random. And so at the very first layer that goes to the brain, the ganglion cells is making averages over a small region of the visual field. And so if the pixels that should be next to each other are not, then I think you'd end up with nothing. I think you just ended up with garbage.
Yeah, right. Well, maybe if you're like, you know, if. If you're. Because, I mean, it's learned in. In early, you know, in early development.
Right. Okay. Modulo learning. In other words, where it might be learned, you might learn something at the high, highest level. In other words, it looks random at the input, but. But once you get to a higher level, maybe it's able to pick things out with the receptive fields are much bigger. That's possible, you know. Well, Stephen, you should do the experiment, put on your randomizing glasses and see how long it takes.
Yeah, right. Let's see how. See how that works.
By the way people have done, they flip the image, they turn it upside down, and that screws people. But you can adapt to that, interestingly.
So let's see, we have a couple other questions here. One is about Robert Heck Nielsen's stuff I never understood. I knew him, and I'm sure you did too.
Yes, he was in the engineering department here at ucsd, and he also had a company, hnc, which was early neural network company for fraud. Credit card fraud detection.
Right, right. I mean, that was a. That was a, a big early AI application. In fact, my first company ended up, after I was no longer much involved with it, ended up going and building what became, you know, one of the standard systems for that.
Okay.
It seems to be an early.
He was an early adopter, but he was also very, very charismatic and had a big influence on the early movement, you know, the neural network movement, because he, he was so enthusiastic and, you know, true believer and so, so what, what.
Kind of neural nets was he using? I never quite got that straight. I mean, I, I talked to him a bunch.
You know, I have to say, I.
Always, a bit, was always a bit mysterious.
He, he, he, well, he was, if you pressed him, he would say, well, you know, this is under patent or something. You know, he has to patent it or something. I've often wondered whether there was something there that I just didn't understand. And I've known several people who actually did have something, but they couldn't explain to me, but they were able to make things work. Mark Tilden, who built robots for Wowee, is an engineer who, he builds things out of parts that shouldn't work but do because of, of the motors, for example, that change resistance when they, when they are run into a block. And now the motor can't turn. That sets the robot off in a different direction. But it's not programmed, it just does it because the motor, the resistance has changed and now so the circuit then flips. But you know, this is, you know, that's another era, another area. But I, you know, I, that's, that's, you know, I'd love to, if anybody out there has actually read his papers and understands them, I'd love to talk to them because I, I, I, I, I've, I've never had enough energy to do that.
The question here about the human brain project, and let's see, I, I, I mean, there are various brain simulation projects. I'm not quite sure what, what this.
Okay, that was the European Union and it was a billion euros over 10 years. And it was the brainchild of Henry Markram, who's an old friend of mine, very good neurophysiologist, but he's also very charismatic and convinced the European Union that he was going to build, in 10 years, he was going to build a human brain.
I thought they were doing, I mean, I remember them, I remember meeting these people and they were talking about, you know, simulating a piece of a mouse hippocampus or some such other thing thing, which seems like not very interesting that.
That was their first step, they wanted to start small, but they, they eventually wanted to, you know, create a simulation of the whole cortex.
Okay, so how far did they get?
They, they. Okay, there's a movie out which actually was a documentary over time which shows how when they started out, they had these grand goals and, you know, and, and gradually over time, they began to realize how difficult the problem was. And even with the he. Okay, so Henry started out with the blue brain project, which was to simulate neural circuits. And his idea was, I just scale this up, right? I've got it working for the small network. I just have to scale it up and I can create the network for the whole brain. And, and so my response to that was, Henry, let's suppose. Let's just suppose. Yeah, that's a difficult problem. But suppose you solve that problem. Suppose you build a really. And he believed in very, very accurate reconstructions of real neurons, not, not just the units, you know, that have no internal structure, but your real neurons. I said suppose you, suppose you do that and you turn it on and suppose that lone beyond it works and you can talk to it and so forth. What have you learned? You know, you, you haven't. It's as mysterious as the brain, right? Yes.
Yes.
Is that really the right gold?
Yeah, I, I remember when people were doing artificial evolution back in the early 80s and they were observing that, you know, this is an, this is a simulation of some rainforest system or some such other thing. But it was an incredibly complicated setup. Again, same same thing. You can observe, know, it's not obvious how you extract kind of the scientific narrative from that. Let's see, there are questions here. Zav is asking. Terry mentioned that current models are about three or four orders of magnitude less compute than the brain, how many order of magnitude and algorithmic development of backpropagation and so on. Does he estimate that we are. So in other words, I think the question being asked is, are there algorithmic. Do we know the correct algorithms and it's just a question of scaling up the amount of compute, or do we not yet know the correct algorithms to reproduce the important aspects of the brain?
So my view is that there are many, many learning algorithms out there. If you give them the constraints that we have on the brain in terms of the units and the connectivity patterns, that ultimately they probably will all converge to the same small set of classes of solutions. And so back prop is just basically takes us there faster. In other words, humans require a whole, you know, development of the brain that goes through many stages of development and takes decades of experience to have an adult brain. Right. And that's, that's learning is going on during that entire time. Right. And, but you can continue to learn even as an adult, maybe not as fast or as, as much. And, and so, so my, my goal is to figure out how that is done even though it's slower and, and, and may take, you know, more resources. Because if we understand that then we're, then we'll actually, I think have a chance to actually accomplish the, the, the, the Henry Marcram's goal. Right. In other words, we, we then it really is going to be interesting because the way that we're going to be able to, you know, instead of feeding data in like, you know, with a shovel, what we're going to do is we're going to have these very large scale networks in the world absorbing data from the world with vision and audition, just like we do. And the brain will develop internally the model just like we do, if we've understood it properly. And so that is where we're going to have end up if you really want to understand how humans. Because we should be able to analyze that process in the computer much better than we can with as we can't. We don't have access to the whole human brain. Right. For all the activity and all the neurons, but we do have access to all the units in GTP3. And so in principle, if we can't figure out how that works, you know, then it's going to be really tough. But I think, I think it should be possible.
Well, I mean the question is whether there is a narrative scientific story of how it works. Yes. Because just saying you run the program and that's how it works is, is not what we have become used to as a kind of success in science.
Well, I think that, I think it's too early to know whether there is a simple narrative. There are some people, you know, I remember when even when I was young people were saying, well, the brain is too complicated to, even for the brain to understand itself. It's not possible. Right. Some kind of. I forgotten what the logic was. It has something to do with it's, you know, you can't, it's self referential or something. You can't understand. The complexity to understand it is much greater than the brain itself. Right. And it's not going to fit in the brain. I, I think that's a bunch of hooey. I think that, you know, and, and Stephen, you've written very eloquently on this, which is the idea of of physical laws being a compression of all of the complexity in the world.
Right, yeah, right. Look, I think the issue is, and we're now getting into. Off into a quite different topic and we probably should wrap up in a moment, but. Okay, we'll reserve it for another time, the talking about the future and let's see, Let's see. Yeah, there's a question here from. It's all gone wrong. Talking about neural nets can solve problems in ways that aren't intuitive to humans and ending up with all kinds of issues. I think one of the things that to me is interesting and surprising is that you make these neural net systems and you ask them to extrapolate basically, or even interpolate, but the fact that they will do that in a human like way, it's not obvious that's going to work that way. I mean, that's telling one something scientifically quite important. The fact that it extrapolates a cat image as being a cat image and so on, and that's more or less like what humans do is quite non trivial. I don't know whether, I mean, you know, presumably the reason it works that way is because architecturally it is the same type of thing going on in the brain as is going on in the neural net. And you know, for the alien, so to speak, that doesn't have the same, you know, the alien's version of optical character recognition, their. Oh, that's really an a. Might be very different from ours because they're, you know, their version of what counts as a small change might be very different.
Yeah, there was this wonderful movie in which aliens came and they couldn't communicate with us because they were using some kind of cloud like things. Inception, I think. No, no, not Inception. What was it called? You were involved in that, weren't you?
What's that? Yeah, yeah, probably a rival. Yes, yeah, that was, that was. They were, they had sort of visual, non linear visual kinds of.
That's right. It was a completely alien, literally. Cody.
Well, even, even if it was actually made in Photoshop. But yes, it was alien. Conceptually alien. It was, they were very nicely drawn. The, the. All right, I think we have, we have a number of questions here about future kinds of things. I think we should probably, I think we should probably reserve the future for another time because I think we could go on talking about the future. For the future is limitless. The history is at least bound.
The history at least has a finite bound. But we've only touched on it, by the way. Okay, so this is something that you know, I live through and you're getting slightly different or maybe very, very different response. But I think though, that we're converging right in. In other words, what you're just bringing up, which is that was the things that we didn't anticipate. I did not in any way anticipate where, what, what, what the neural networks were able to do. I didn't. Had no idea how they would extrapolate in scale.
Right. Nor did I.
It was a big surprise and, and in fact, it continues to surprise me as new things are coming up every year. There's, it seems like there's some breakthrough, but we don't know. It's, it's, we're in the middle of it. We'll see.
Right. But I think, you know, the, the, the thing that, you know, I didn't really believe in neural nets for a long time. I, you know, it seemed like they were. The things people were writing, I have to say, just seem very complicated, very arbitrary, you know, and, and all these kinds of different methods and so on. It just didn't, it didn't seem to be a coherent kind of scientific set of things to be talking about. And so it's, I mean, the fact that suddenly the engineering has started to work is, has been surprising. And it's, you know, I'm just curious. For somebody like you, the thread of belief in your case is probably linked to the fact that you're interested in how brains work. I mean, had you been purely interested in the technological side of it, maybe you wouldn't have continued to be a believer.
Okay, here was my belief and my intuition, which I think Geoff Hinton shared, which is that the only existence proof that any problem, vision, intelligence, could be solved was the fact that nature solved them. Right. Somehow. So why not look under the hood and try to extract some principles, general principles, like massively parallel, like a high connectivity, like learning. Right, Basic principles. See how far we can get. Right. That was a very, you know, it seemed obvious to me back then. Of course, I didn't know how long it would take, but you know, it was just a matter of believing in nature.
But, you know, there's a certain sense in which, you know, there's molecular scale stuff going on in the brain. You know, people say, oh, we can only explain brains by thinking about quantum mechanics, all these kinds of things. You don't know how low you're going to have to go. And the fact that I think one of the things that's really very interesting about Chatgpt is It kind of tells you that a large chunk of the kind of language and language related thinking really can be done at the level, at this pure computational level, that there's no mysterious infrastructure underneath there.
Right. So I think this is an interesting observation and I think language is very recent. It's only within the last hundred thousand years. So it couldn't have evolved a language organ, which is what Chomsky believes. Right. It had to have used all of the machinery that was already there, may be using it in a different way. And it's all about sensory motor loops and controls. And I already told you about the basal ganglia playing a role in sequences of actions. So if the brain was already set up for it. Right. And natural. And because language maps so well onto the brain, the machinery that was already there means that there's some principles that language had to be using, similar principles that say vision was using, or auditory processing, speech recognition.
Right. By the way, is it the case that the timing of words coming out. If you look at your loop in this basal ganglion cortex loop, can you measure, oh, the time it takes, the signals.
It takes about 100 milliseconds to go from the cortex of the basal ganglion back to the cortex again.
And how many words do we say in 100 milliseconds? We say, we say a word or two in 100 milliseconds.
Okay. I, it depends on the words. But, but I'm, I'm, I'm thinking that it's about, you know, ten words a second. You know that I, I think that's pretty fast. Yeah, that's talking.
So you're saying every time we do one of those loops, we make a word.
I, I think, no, I don't think it's a word. It could be just a phony. Maybe it's just a sound.
Well, like a token in chat GPT. Yeah, like it's making one token just like it is.
Right, right. Maybe that, maybe at that level.
So you think that, I mean, you think it's plausible that the outer loop of human language is that loop, which is the same type of thing as.
You know, I don't, I can't imagine it being anything else. You know, I, you know.
Loop is.
What we use for doing things like sports and very complex thinking.
Okay, so let's assume you're right, that language, which I'm sure is true, that language is, is simply sort of recruiting existing characteristics of the brain.
Right.
The question is, okay, so you say language is invention. In the last 100,000 years, one could discuss whether other organ, you know, other critters have, have some form of language. But let's assume it was invented fairly recently in us, so to speak. The question is what other things can we recruit from other parts of the brain to have quite different kinds of modalities of action? I mean, language has proved to be pretty useful because it's a way of communicating abstract ideas from generation to generation, so to speak. And so a question would be, if we try to imagine, know, for the, for the aliens that have the same basic brain architecture, but they've recruited two more, they've invented two more general ideas, what would those gen do we have a clue about what those general ideas might be?
Well, now you're talking about the future.
Okay, okay. Now I'm talking about.
Okay, but, but, but, but, but here's something to keep in mind, which is that language is imperfect. It's very slippery. These concepts like intelligence or understanding which people are arguing about, they say, oh, GPT3 can't possibly understand what it's talking about. Right? It just sounds like it understands what it's talking about. What do you, what's understanding mean? How can you define, how do you measure it?
Well, I mean, for me, because I've spent my life kind of building computational language, you know, for me, it's. You have this thing. Can it be evaluated? Can it be, can you compute from it? That is an operational definition of saying that you've got this representation of something that can actually in language. That is not what we do in language. We are just sort of building pieces on top of what we have said. But if we are thinking about computational language, where you've actually taken the utterance and you've converted it to something where the computer can go off and execute something from it, that is an operational definition of understanding. It's not the one that mostly it's not the one that we use, but that's an operational definition.
So there are many. That's a very good one. And there are probably many other definitions that other people use. That's why they get into arguments, because people have different definitions when they're trying to understand something themselves.
Well, I think the point you're making there, in effect, is that what we do in computational language and the way that we evaluate things, execute things from that is something brains actually don't do. And potentially, you know, and so I suppose that's an answer to this question of okay, so what else could you know? If we've invented language in the last hundred thousand years, what could we invent. And then the question is, does the architecture of the brain support. Does the, you know, architecture of computers and so on does support executing computational languages? That's obvious. The architecture of the brain supports that. But, yeah, now we are indeed talking about the future and we're talking about.
And we can go on for many, many more hours. So, you know, let's plan to get together again sometime.
Yes, indeed, indeed. And did we miss, I mean, you know, in terms of the history, you know, you talk about other perspectives, but I think we covered, you know, from what I know of this history, I think we now, you know, I know some of this history I know through you, but some of it I know independently. And I think we, we, you know, we, we hit most of the, most of the big topics that I'm, I'm aware of that, that, that, that came in there.
Yeah, there are many, many side pathways that, you know, many, many more people should be given credit. I, I, I have to say that, you know, I've, I've, I've focused on people I know and people I've worked with and so forth, but so many other people out there that, in fact, I can't keep up with the literature anymore, you know, and coming out of Arxiv, it's just overwhelming.
Well, right. I mean, you know, this is a field which has gone from a tiny group of people during its sort of, you know, lean period to, you know, it must have gone from tens of people to maybe even a hundred thousand people or something at some point that.
You know, I was talking to Eugene Vigner once. He was at, taught me statistical mechanics when I was at Princeton. And I asked him what it was like to be there, you know, in Europe when quantum mechanics was being discovered. And he said that. Well, he said, well, first of all, back then, physicists didn't have the kind of prestige it had back then. It was, in fact, you, you, it was. Very few jobs were available at universities and you couldn't get, you know, it wasn't very lucrative. And his father told him that he should get a job in chemical engineering, which he did. Okay. And he said that the number of physics, theoretical physicists, now, theoretical physicists were so small, you could know, you could know them all. He would write letters back and forth, right? And that was, that was how the whole thing got off the ground. And maybe that's the way all things get off the ground. It's never, you don't start with a mass of people. You start with a small group that actually believes in what they're doing, and they're willing to give up a lot of prestige or whatever money to do it.
I think this is a weird case. I think neural nets are a bit unusual because I think that as we were mentioning earlier, there were all these separate little pockets. I think that's not so common. I think many of these things, the tree grows from one trunk, so to speak.
Ah, okay. That's an interesting observation. Yeah.
Because I don't think in quantum mechanics there were not multiple trunks. It was not like there were different pockets of things going on. It was pretty much just one. But I think with neural nets, what's strange is that the original sort of McCulloch Pitt style initiation is the seed, I think, from which all the other stuff grew. And yet there were all these separate branches and the same seed, but it wasn't a single trunk from that. And that, I think is unusual. And I think that the timescale, also the time scale from the initial seed to fruition is extremely long. I mean, many generations.
Okay, so one possible explanation for that, I agree that it was very diverse, but one possible explanation is you have a much bigger search space to explore, and many groups were going off and searching different parts of computation space. Right. Cojones and Stu Gan and so forth. And, and you know, it took a long time for all of those to sort of be unified into a large scale. What was called machine learning for a long time. You know, Neurips was, was the biggest machine learning conference and the most important. Now it's been rebranded as an AI meeting. Although it's still, it's still. We're. We're talking, it's still neural networks, which is what we started with, but it's gotten much richer and much, much more sophisticated in terms of computational understanding of graphical models and things.
It is so bizarre that the original neural net idea from whatever it is, you know, what is it, you know, 85 years ago or something.
Right. Is still.
That's what's running inside ChatGPT. I mean, lots and lots of copies of that.
It was. Yeah, that. That's shocking. Shocking, you know, that it could have have built. It could have built from simple ideas that were floating around back then and coalescing and. But by the way, the, the amount of resources that went into these large language models is staggering.
Yeah, right. Well, that's a whole different.
$10 million to, to learn a single network. Right. Oh, my God. You know, that's.
We'll see whether maybe the brain has figured out much better ways to do it. And that.
That.
Right. It's all right. Well, we should wrap up here. Thank you all for. Thank you, Terry.
You're welcome. Welcome, Stephen.
Interesting conversation. And thanks to folks who've been listening here. And we shall see you all another time. Another time we. We meet and talk about the future.
Back to the future, indeed.
More from Guy
Cheryl Strauss Einhorn | The Human Edge: Smarter Decisions in the Age of AI | Talks at Google
Rachel Glennerster | Innovation and the Center for Global Development | Talks at Google
Rebecca Winthrop | Rethinking the Purpose of Education in the Age of AI | Talks at Google
Dr. Robert Wachter | A Giant Leap: How AI Is Transforming Healthcare... | Talks at Google
