#214 Learning & Memory, For Brains & AI, with Kim Stachenfeld, Senior Research Scientist at Google DeepMind

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com the way that we think about the role of episodic memory, of rapid memory contributing to learning in the brain is actually quite similar to how we think about retrieval, augmented generation like RAG in these language models that if you give them access to a memory buffer in which you can just put entire new experiences that they could then draw and add to their context. And that's actually in a lot of ways very similar to how we think about learning in the brain. And I think both neuroscience and AI could find that an interesting and fruitful analogy.

Welcome to Data Framed. This is Richie. There are three ideas dear to the heart of this podcast that we typically think about separately learning things, teaching things, and messing about with AI. There's a scientific field that touches on all three of these, namely neuroscience. As an intellectually curious human, I'm always keen on hearing tricks for learning more effectively and, well, not forgetting things all the time. Secondly, as an educator, I want to know how to teach better. And thirdly, as an AI nerd, I'm curious as to whether there's any overlap between how humans learn and how AI learns, and how whether neuroscience research is going to result in better AI for us all. Today's guest is Kim Stackenfeld, a senior research scientist at Google DeepMind and an affiliate professor at Columbia University's center for Theoretical Neuroscience. Kim's research involves reinforcement learning, the hippocampus, representation learning, and graph neural networks. That's a lot of technical words for saying she's at the intersection of brain research and AI research, and so ideally poised to answer our questions. Her work has been featured in the Atlantic, Quantum Magazine, Nautilus, and MIT Technology Review. In 2019, Gim was named one of MIT Tech Review's Innovators Under 35 for her work on predictive representations in the hippocampus. So let's learn about how our brains work. Hi Kim, welcome to the show.

Hi Richie, thanks for having me.

So I'd like to start off talking about learning. So how do brains learn?

It's funny, when I first, like, my first reaction to that question is, oh, gosh, how could I? Like, that's a big question. I think when I was thinking about this a little bit in preparation for this, I was thinking two things do really stick out as special things about how the brain learns, especially in comparison to modern AI. And that's the way that it's structured. One way it's structured is just developmentally. Cognition emerges gradually. Babies are very different from adult humans to just start out with the obvious. And humans acquire skills at different stages in their life. They gradually start to experience more and more of the world as their brains become prepared to deal with it. They first learn how to make sense of the world with vision and audition. They start to form memories. They start to form speech, motor control, social behaviors. You've probably heard the brain is not fully developed until you're 28 years old, often in the context of, like an argument with a parent when you're a teenager disagreeing about what you should be allowed to do. And it's true. It takes a while for cognition to develop. And we have experience that's age appropriate as we go through life. This is really different from how language models are trained, where it's just right away, they start seeing random samples from the Internet, and slowly, just the entire data set is presented to them at once, and they very gradually start to make sense of it, rather than building from more simple things to more complex things or building from certain skills to later skills. Another thing that's really special about learning in the brain is that it seems like there is some regional specialization, some modularity to how learning is structured in the brain. We have theories about how different brain circuits in different brain areas contribute to different types of learning, which all mingle together in a really complicated and in some ways still hard to describe ways. I focus, for instance, a lot on hippocampus. Hippocampus seems to be associated with rapid memory for experience. So if you, for instance, summon to mind the vivid experience of when you parked your car this morning and which parking lot you decided to drive into and can kind of summon that and almost re experience it, that's episodic memory. That's very different, for instance, from the more gradual process of learning from reinforcement, gradually learning which actions are rewarded, which actions are not rewarded, and modifying your behaviors over time. And there's evidence that there's a lot of specialization and these processes kind of happen somewhat separately and then all joined together. There's also just like tons of other kind of learning. Those are two I've thought about a lot, so I mentioned them. But cerebellar learning, cortical learning, there's just all sorts of. It's very specialized as opposed to the kind of monolith that large neural networks are.

That's absolutely fascinating. There are lots of different types of learning and it's fairly reassuring, I suppose, that there are lots of different ways you can learn things. It was also kind of fascinating that you mentioned reinforcement learning, which is also a technique for training AI. So it's good that brains and AI have some overlap.

Oh yeah, yeah, lots of overlap.

Super. I also found fascinating that you said that your brain's not fully formed or you continue learning things up until you're 28, which is much longer than I expected. Like you sort of think once you're a grown. Yeah, basically you've done growing with things. It's interesting that carries on a bit longer. Since you mentioned there are lots of different types of learning and some of them a bit more specialized. Can you talk me through? Does it work differently for learning different kinds of things?

A classic neuroscience example of this is the difference between episodic memory, which I'll define, and procedural memory. There is a very classic study in neuroscience, actually. My brother in law is a medical student and when he was taking his neurology class class and had his class on the hippocampus, I said, I bet you $10 your very first slide is going to be this guy Henry Molaison, who had both of his hippocampi removed and lost the ability to form new episodic memories. That which he did not take. But I would have won, basically. This man, Henry Molaison, unfortunately was experiencing seizures which originated in his hippocampus. They removed both of his hippocampi and he entirely lost the ability to. To form new memories. He would experience things and then just as soon as something came would happen that distracted him from his current experience, he would forget what had been previously going on. So he was very limited in his ability to learn new things. However, I think that he could eventually learn with repetition was complicated motor behaviors. In particular, they used this thing, a mirror star tracing task. You watch yourself tracing the outline of a star in a mirror. It's something that's kind of tricky to do. And he would practice doing this, have no recollection of doing it, but gradually get better. Just surprising himself that he was so good at it every time he did it. So that's kind of one of the really foundational results in neuroscience about how there are different learning systems and you can really severely affect one while leaving others somewhat intact, Although they seem to relate to each other. Presumably, if he had had a hippocampus, he might have been able to learn that task a little bit faster. These systems interact, but the fact that there is as much modularity as there is is often still surprising to people.

Well, first of all, it sounds like an absolutely horrible experience for that man, like bits of his brain not working or not even present.

In the early days of neurosurgery, there were a lot of kind of reckless surgeries. I'll say we don't do that anymore.

Okay, that's also very reassuring, but, yeah, very fascinating that there are these independent learning systems. So even if you lose one ability, you can still learn things in other ways and talk about learning things in other ways. A lot of our audience are very interested in learning things about data, and you want to be able to learn faster. So do you have any tips for how to learn more effectively that are actually backed up by science?

Yeah, I'm not a medical doctor, so, you know, grain of salt. It seems generally that the boring things people recommend, Sleep, exercise, social interactions, repetition, those are pretty useful for learning. I think one thing that's particularly interesting from the perspective of somebody who studies hippocampus and reinforcement learning Is that some of these have been associated with changes in neural circuitry that seem a little evocative. Hard to say for sure that the observations made at the clinical level really relate to what people see in the lab. It's a complicated system, but exercise in particular is associated with neurogenesis in hippocampus, the creation of new neurons. Social interactions also seem like they have this effect, the sleep. Also, there's a lot of theories about how sleep is useful for converting experiences that you've had recently into knowledge that's more general and more robust and will stay in your brain for a longer period of time. There's a theory of complementary learning systems, this idea that hippocampus helps you learn things really rapidly, and eventually patterns from these experiences are extracted and distilled in your cortical circuits. And sleep is hypothesized to be really important part of that process. So, as somebody who doesn't sleep that much and is loathe to exercise, I find that's a little depressing. But, you know, it is nice that the things that that kind of lead to a happier lifestyle also can be justified in terms of promoting brain Health. What I also find really interesting is just that novelty seems like a really big driver of the formation of new memories. There was a lot of concern during COVID that people weren't just getting enough memories, novel sites, they were staying in their house too much and not seeing new things. And that seems like having a persistent experience of novelty is important for laying down new memories. That's also kind of something in the realm of the things that kind of make you feel happier seem like they also put your brain in a good state to learn stuff.

The idea about sleep is particularly interesting because quite often you see on LinkedIn people talk about, oh, yeah, I get up at 4am and I'm super productive immediately. Yeah, I'm like, well, actually, maybe you should probably have had a lie in. You'd be smarter.

Yeah, I mean, if they're going to bed at like 8 or whatever, that's probably a pretty solid plan. I mean, having a good schedule is a good way to get more sleep. Having a schedule that deprives you of sleep is probably not. I mean, I don't know of any science that support. Just to say it in careful terms. I don't know of that being supported by anything. I think, if anything, there has just been lots of research recently into how sleep. It matters a lot more than we think. And there's also some really interesting cognitive theories of how that works. So, yeah, I would say sleep more than you want to to your listeners.

I'm gonna try and see if I can get away with having a nap at work just to say, oh, yeah, it's important for my brain power.

Yeah.

All right. So just relating this. So we talked about how to learn more effectively. And related to this is like, you learn stuff, you forget it. How can you improve your memory?

I don't know if you've heard of the idea of memory palaces. When I had heard them before I learned any neuroscience, I was like, that seems a little bit silly. The idea is basically you imagine a spatially familiar place. Sometimes people like to just construct them. But I think the classic example, you use your childhood home or something, and you can memorize random sequences of things by kind of binding them to different locations in your memory of your house. There's actually been like a decent amount of research into this. It seems to work quite well. Other things that boost. They don't boost the skill of memory necessarily. They're more like a strategy for remembering particular things. You can bind them to spatial arrangements, as in the memory palace. You can bind them to songs like, I Learned a song in Spanish class in high school to remember all the irregular verbs. I think that's a pretty common trick. And then another general thing in that family is you can put them into a story. This is sometimes what people who do those memory games where they try to remember random sequences of cards, they construct a story about it. Why these things work and the extent to which they really work in systematic ways is a bit of an open question. But there's a modeling framework that make some interesting hypotheses about it. I think the basic idea is that we often have the experience of applying a familiar narrative structure to a new situation. If I watch a rom com, I know generally how rom coms go, and I would be really surprised if something happened. Besides the characters ending up together, things tend to go in a particular way. I know how walking through my childhood home is gonna go. I would be really surprised if suddenly the garage was in the wrong place. I know how songs are going to go. And if all of a sudden I started hearing white noise instead of a song, you would be very surprising to me. These familiar sequences can sort of form a template to which we can bind new things. That's at least the hypothesis for the aficionados. There's some really nice computational work from James Whittington, like implementing this idea in mathematical concepts. Elafite's group also has some nice work, like implementing this specifically in the spatial context, although it's thinking more about how animals navigate than it is all of memory processes. But it's a really general framework and can be thought of in that way. So I think an emerging idea in computational neuroscience is that these templates can be useful for binding arbitrary concepts. And in general, I think this is one of the reasons this is just me kind of speculating. But why curiosity is a really good driver of learning and memory is if you're actively trying to put new knowledge into a story, to stitch new things you've learned into other things and make relationships, you are naturally leveraging this process. You're trying to kind of apply the templates you know about how things logically fit together and using that to scaffold your learning. And then you'll have better memory for it because it's both more interesting and more salient and wakes up more of your brain, but also because it is leveraging structure that's already in there and binding to new things.

That's brilliant. We do spend quite a lot of time on this podcast, talk about storytelling for improving the impact of your data analysis and things so it's good to know that is really backed up by science. The other idea about songs, that's also terrifying that I'm gonna have to start singing songs about data just to make people remember them. Yeah, it's a skill I don't have, but maybe for someone else. So on the flip side to this, there are always people come up with like advice on how you can learn things better or remember things better. So are there any things that don't actually work that are maybe in popular culture?

There probably are some that I'm not even thinking of because I just kind of put them out of my own brain and then forgot that they're out there. Two that came to mind are things that show up in my social media ads a lot are games to improve cognition and supplements. Games challenge your brain in different ways and I'm sure there's like ways they can help. We don't really understand it fully. So definitely a lot of those things which are like play this app and organize luggage on a phone screen into different shapes. I don't think that there's a ton of evidence that that makes you better at anything besides playing that particular game. And it definitely will make you better at playing that particular game. It's hard for me to imagine that you're better off playing a game on your phone to really get better at learning and memory. And that's better than having a conversation with another human being, which requires you to recall things from your memory and improvise and adjust in a very adaptive way. Lots of games really require a lot of thinking and recall. I love doing crossword puzzles and I like to think that there's some benefit to them. Although besides being more likely to use words that have a lot of vowels in them, in my day to day speech, it's hard for me to know for sure what they are. I'm playing games like chess and stuff. It's hard for me to imagine that doesn't build up cognition. But yeah, I think a lot of these games over promise and it's good to basically just know that whatever the claims are, it's probably just in the realm of uncertainty relative to neuroscience supplements. Also just be careful with those. I mean, that's almost more dangerous because there's definitely a lot of pills you can take that will change your neurochemistry. If you have ADHD or depression or something, one could talk to a psychiatrist and get diagnosed with drugs that will definitely have an effect on your focus and learning and memory and they have side effects. And one shouldn't do it lightly, but there's definitely an effect medicine you can take can have. It seems like there's also just a lot of other stuff which is advertised and I have no idea what it does. A heuristic I often have for evaluating those things is I look at the about pages and see who is associated with it and see if any of them are doctors and professors. It's a hacky method. That's not a surefire way to know that something is or is not a scam. But it's at least one source of evidence you can draw to see how aligned it is with what medical and scientific institutions are generally supporting.

So maybe don't just buy random pills off the Internet.

Maybe that's what you are the obvious. Like don't just take pills off the Internet and then it could go poorly.

Yes, that's good life advice in general, I think. All right, and let's talk a bit about about the overlap between brain research and AI research. So can you talk me through how does your neuroscience research relate to AI?

Yeah, historically the fields have had a lot of overlap. Thinking about the problem of how to learn from reward or how to make sense of things that you are looking at the computer vision problem or how to implement memory. Well, these are just problems that show up in both AI and neuroscience. And historically there's been back and forth conferences like rldm, Reinforcement Learning and Decision making, which are very interdisciplinary and involve researchers from neuroscience and computer science thinking about the same things. There's always debate about was this idea from neuroscience or did neuroscience get the idea from AI? I don't know. I haven't been alive for the entire history of neuroscience and AI. It seems like it's an efficient use of intellectual resources for there to be some people who are just thinking about the same problem in both the neuroscience and AI context that it makes. Because these problems have a lot of overlap. Ideas people think about in neuroscience, they have relevance to how they work in AI. Ideas people think about in AI constitute nice hypotheses for how the brain works. It's quite possible that a random idea that we have for an algorithm may be a really bad hypothesis for how learning in the brain works. There's very different constraints that a system that works with GPUs has to obey versus a system that has to form itself out of a single cell and also learn and be subject to biological constraints and be made out of food. There are very different constraints on these systems, but they have some computational problems in common. And thinking about what's in common between them, I think is just a good use of researcher time. That's kind of how I think about it. The problems have a lot in common and it makes sense to think about both in the same space so you can make double up on your efforts.

It's cool. There's a lot of overlap. I'm just wondering, have you got any examples of maybe an idea you've taken from AI or how you've used AI as part of your research?

Yeah. So I've done a lot of work in particular on prediction in the brain and how memories are used to make predictions about what's going to happen them and how those predictions are used by the brain. This was really building on ideas from the field of machine learning, or AI, and testing the extent to which these explain activity that we see in the brain, in particular in hippocampus. There's a lot of work in AI on how making predictions can be useful for different things. One thing is just the very act of planning involves making a prediction about what's going to happen and, and how it depends on your actions. If I want to figure out what time am I going to get to work, I might just kind of have some table of experience that I've remembered, like, oh, if I leave at 9, I'll get there at 9:30. If I leave at 8:45, I'll get there at 9 15. Or I could play out in my head like, all right, I'll go through this sequence of steps and that'll sum up to about 30 minutes. Selecting Actions is often informed by our predictions. Also, just the act of trying to make predictions often causes you to learn about things you might otherwise not bother learning. If I'm trying to predict what will happen, I'll have to pay attention to all these features about my environment that have information about what's going to happen, for instance, like how much traffic there is today or if it's raining or other factors that could affect commute. So this has been a big topic in machine learning, how models can learn to make predictions, how making predictions can cause models to learn better representations of their environment. And one area of research that I've worked in a lot is how we can think of hippocampus is contributing to making predictions about what's going to happen next, how hippocampus forms memories of what has happened and uses that to support predictions about what's going to happen. You can kind of relate the past to your present in the same way that your present relates to your future. So memories can be used to forecast what's going to happen. We've used a couple different ideas in machine learning. In particular some ideas called successor representations, about how to represent your observations in a way that incorporates information about what's going to happen next and how this captures some ways that cells behave in hippocampus, how they change their firing patterns and their activity patterns over time.

I never really considered how hard it is just to figure out how long your compute is going to take. Essentially there's different approaches. So you can either go based on past experience or you can try and reason about it from scratch. So actually this is kind of interesting because you think about particularly like with these large language models, they're really great at predicting like what should the next word be, but they're rubbish about reasoning about things. Do you have a sense of why prediction is easier than reasoning in this case?

It's hard to say objectively that they are rubbish at reasoning or not rubbish at reasoning compared to their ability at prediction. Because we don't necessarily have a frame of reference. I think we're broadly impressed by their prediction abilities and maybe less impressed by by their reasoning abilities. But it's hard to really evaluate this in a quantitative way. And I have some colleagues who did some nice work on this. Andrew Lampaden and Ishita Dasgupta had a lovely paper comparing reasoning failures in large language models to reasoning failures in humans. Because humans make a lot of mistakes in reasoning too. And they basically showed that for reasoning problems that were more familiar, that were congruent with experience, language models tend to be better at it. And humans show the same effect. One example they had is if you have some task where you have decks of cards and they have pictures on them based on the picture, you have to make some judgment about what's on the other side of the card, which is a number. So for instance, if you have a task like there's a picture on a card and then you'll see a number, and based on the picture, you have to decide if the number is greater than or less than 21 or greater than or less than 17 for another image. That's something people can learn and language models can figure out, but they make a lot of mistakes. It's hard to keep all that information in memory. If you make the pictures, a picture of a beer glass to represent the drinking age and a picture of a car to represent the driving age, people are much faster at it. They make fewer mistakes. It's much easier task because we're Used to making that kind of judgment, they refer to it as amortized, that you've already sort of done some of the work to learn about that computational problem and you've already practiced making that judgment. It's familiar to you. I think there's probably a similar thing in language models that they're trained statistically. They're trained to predict what is the most probable next word. And there are reasoning problems that are implicit that have happened in the corpus that they're trained on that exist in that language data, so they have some ability to do it, but they're trained statistically and reasoning problems that are really far away from their experience, which are the things we really think of as canonical, exclusive reasoning problems. Problems that are only reasoning and not just memory, not just relating to your prior experience. Those are things that are unfamiliar to them. And this really gets to the challenge that a lot of deep learning models have of generalizing out of distribution, of handling really unfamiliar data. And I think one thing about this you may have talked about on the podcast, chain of thought reasoning or chain of thought prompting. Famous example of this is if you prompt a language model, let's think this through step by step, it'll give you a longer and also more accurate answer. And there's some work from Noah Goodman's group hypothesizing that the reason for this is that it forces the model to break an unfamiliar problem down into familiar parts. Experiences is local and you have experience with individual steps of reasoning, but you don't have experience with this particular reasoning problem. So when you convert a problem that is new into a set of relations between familiar parts, you can start to take something that's unfamiliar and recast it as something that's familiar that you know how to deal with. So I think probably the way humans are good at this is that we have experience with this reformulation process and can do it kind of automatically. And the extent to which we are good at this is also kind of something that psychologists are constantly trying to assess and a good reference point when thinking about how machines do stuff.

This actually seems like good advice for people in general. It's like if you don't know how to solve a problem, then just break it down into small problems that you can solve. You said that familiarity makes things easier to reason about. Are there any other things from neuroscience where you think, okay, this might be useful to apply to AI to help AI reason better?

Familiarity makes things easier to reason about, especially if you are a neural network, where neural networks are so expressive, they're capable of learning such a broad array of functions that what they're going to do on data they've never seen before is kind of unconstrained because they could learn to do something different on that data if you wanted them to. Their expressiveness is deeply tied up with their ability to learn complicated things. Their expressiveness is tied up in their what they might that we don't know what they're going to do on unfamiliar data. So recasting things as familiar is particularly important if you want a statistical learning system like a neural network to do a good job on reasoning problems. A thing that I think about a lot from the perspective of augmenting the current state of the art large models is the role of memory and how it's different in humans. The way that we think about the role of episodic memory, of rapid memory contributing to learning in the brain is actually quite similar to how we think about retrieval augmented generation like rag in these language models that if you give them access to a memory buffer in which you can just put entire new experiences that they could then draw and add to their context. That's actually in a lot of ways very similar to how we think about learning in the brain. And I think both neuroscience and AI could find that an interesting and fruitful analogy. One thing I really like about Andrew and Ishita's paper on comparing language model reasoning to human reasoning is that it demonstrates that one lesson for thinking about the relationship between AI and neuroscience or AI and psychology is that it's really hard to evaluate how good a model or a brain is at doing stuff. And the entire field of psychology has really like spent a lot of time trying to really tease apart how the brain solves different problems. It's difficult to quantify and to systematically say things about how an intelligence system is working. So I think that using the brain as a reference point is a really useful way of calibrating expectations. Should we say this is good? Should we say this is bad? Like using humans as a reference point is really helpful for just calibrating our expectations and then the processes by which we tease apart how the brain works, which is the job of neuroscientists, or how behavior works, which is the job of psychologists, has a lot of complexity to it. And I think there's a lot of interesting reuse of methods that will eventually go into that.

You mentioned that it's difficult to quantify how a brain does all this sort of learning and memory and things. Should we be using different metrics for AI then? Should we just be saying, okay, this should have a completely different system to it. How do you think about that?

A lot of what goes on is that people will play around with language models and then post something on Twitter, which is a single example, and then people will read into it and post it around, and then somebody will try prompting in a different way, and you don't get that same effect anymore. This approach has pros and cons, honestly. I mean, I think there's something clearly anecdotal and therefore not super scientific about it. However, it's almost kind of like field research. It's lots of people playing around and trying different things and finding things, and those constitute good hypotheses for what kinds of things we should look for. I think the mistake, perhaps, is confusing single examples with conclusions rather than using them to inspire hypotheses. You hear a lot about the scientific method, which starts with a hypothesis and then constructs an experiment to evaluate it. Where these hypotheses come from is a little bit of a fuzzy story, and people trying different stuff, people having experiences in their lives, people just kind of think in random thoughts is a big part of that historically, I think. And some hypotheses just strike people as interesting questions and some don't. And I think that this process is really good for generating hypotheses. Actually evaluating them is tricky. There's a lot of work that goes into this. I don't necessarily want to say people are doing it wrong. People are trying really hard to come up with evaluation systems to try and quantify what these models are good at. If they're good at reasoning, if they're good at math, if they're good at critical reading comprehension. They are using tests that we use on humans, which is very much kind of building with a psychological approach. And then when they pass this test, then they start to think, well, wait, where does that test fall? Let's try to make new tests as a result of it. I don't think anybody's doing anything fundamentally wrong. It's just a hard process. I think that's one of the lessons, is it's just really hard and requires a lot of investment to come up with good evaluations of how these systems work and to really, like, calibrate expectations and test hard hypotheses about how models are working.

I like that coming up with hypothesis. One of the things just having random thoughts, that makes you feel good. Oh, yeah, maybe that idea that just popped into your head, probably write it down. It could be something interesting.

Yeah. One thing I Found really vexing in graduate school, but now find it kind of fascinating for me is that the scientific method starts with a hypothesis and no one tells you where to get a good hypothesis. It's almost an aesthetic question of what is a good research question. Sometimes there's obvious examples of things that are interesting, but it's really hard, I think, to go from what the kind of question that motivates me is something like, how does memory work in the brain? That's not like a hypothesis, that's like an interest topic. And then a specific hypothesis like, oh, it's using, I don't know, k nearest neighbors lookup or something like a specific algorithm that can constitute a hypothesis. Why is that the right one to start with? Is there evidence that it's even in the vicinity of that? What if it's just something entirely different? And this question doesn't really get you leverage? It's very concerning. And I think one of the things I like about this AI inspired neuroscience is it's a nice batch of hypotheses. These algorithms that work well can comprise a library of hypotheses that we can evaluate against data in the brain, which at least aesthetically, is an area that appeals to me. Although maybe it doesn't come down to something fundamentally easier to justify than that.

Systematically coming up with good hypotheses maybe an unsolved problem at the moment. So future of research and speaking about research, I know one of the things that you work on is to do with associative memory. Can you tell me a bit more about what that is?

A topic that I have worked on a lot is how to think about how relationships are learned and represented. This is a framework that has been around in psychology and neuroscience for a while. The idea that humans and animals are learning relationships between things. And we can gain a lot of foothold on how learning and reasoning work in the brain by understanding how the brain can implement this process of learning about relationships. So relationships, it's a pretty broad term. You could think about relationships between people. I guess that's the most colloquial usage you can think about. Spatial relationships, semantic relationships, like da Vinci's painting of the Mona Lisa is in the Louvre. Each of those represent relations between different entities that that live somewhere in your mind. Knowledge graphs are a data structure that's kind of generally used to represent these kinds of associations. And there's been some work hypothesizing and testing the idea that humans and animals learn associations and that this kind of associative learning Is a very flexible and powerful basis of memory. And we can think of how hippocampus is forming new memories as building associations between entities that were experienced together. And part of the power of this idea is that it gives you a way to go from specific experiences to more general concepts. If I, for instance, learn that da Vinci painted the Mona Lisa and the Mona Lisa is in the Louvre, I have learned something implicitly, that da Vinci has a painting in the Louvre. The ability to learn these little things and then chain them together into broader inferences Is a really important concept in just how we learn more complex structures than we started out with. How we can learn bootstrap what we've learned in order to make broader inferences. A particular example I've worked on a lot is thinking about the relationships between spatial relations. There's been a lot of work on how space and maps and navigation is represented in the brain and how other kinds of associations that more generally just relate to two different things you can experience in time relate together. I think some of the work that I've done can really be summarized as thinking of how we can think of spatial navigation as part of the more general problem of how we learn Statistical structure about how events are joined in time. Space is one way to constrain that. If I enter my building, then I can now access all these other states in the building. They're all now states that I can predict as coming up in the future. But there's other things that scaffold this too. Like, you know, if I start a movie that's a rom com, I can predict at the end that the characters are going to end up together. And being able to think about all of these in the same framework lets us think, how can we take computational models that have been applied to understand how a rat solves a maze and use that to think about how episodic recall can work more generally, how memory systems can support planning and reasoning a little bit more generally.

So this seems to come back to that idea of the memory palace you mentioned earlier, where you're associating a specific memory with a physical location. You mentioned that spatial associations and temporal associations seem to be similar things. So are the differences between them, or do we sort of reason about these things in the same way in general?

Yeah, there's different hypotheses about it. I guess I should say the work that I've done is hypothesizing that these are really part of the same process, that you can either think of spatial environments as constraining temporal Relationships like, you know, you can't teleport, so things nearby in space are going to be related in time as well. You can also think of it the other way. And that's really more like the modeling framework that we've had, that if you have a system that's trying to learn temporal relationships between things, one thing it will automatically learn is that nearby places in space are related because they tend to appear nearby in time. So one thing we looked for is there's been some experiments that try to decouple this. So for instance, there was an experiment from Kristen Doler's group where they put people in VR and had them navigate around and then added teleportation, which decouples space and time, and found that this had an effect on representations in hippocampus in humans. So that's one of the kinds of things you can model by thinking about this, is that if you're generally learning just more about temporal relationships, you get spatial relationships for free. Because space constrains how different states will be related in time. But you can also get other stuff too. And because hippocampus is thought of as this general memory system that has a special relationship with, but also can remember, be responsible for learning experiences that have nothing to do with space, adopting this more general sequence formulation seemed compelling. There's other work. Yeah, I think this is kind of a big field now, is thinking about the relationship between statistical learning and spatial structure. We weren't the first to do work in that space, but we had a particular computational model that we used that really leveraged this idea. And I think the kind of counter hypothesis is that these areas are more space specialized for spatial navigation and you get some temporal stuff for free because space constrains navigation, but that it's really more specialized rather than just all coming out of the same general purpose algorithm.

You mentioned the idea of combining statistical learning with associative memory. So this seems a little bit like this idea with AI where you combine the large language model, just the prediction very well statistically, and then you have a graph database or some kind of knowledge graph to handle the reasoning. Is this a similar sort of idea?

It gets really similar. Yeah, I think that's a really good example. I alluded to a retrieval augmented generation at one point where you have a predictive system and that's augmented by specific passages that you might recall from a database or specific pieces of evidence that you might recall from a database and use to inform your prediction. I think this is a really similar idea. This particular structure you're retrieving from might have graph structure about how things are related to each other that can inform the retrieval process. And yeah, I think it's really quite a similar concept in that it's combining these ideas about understanding how entities are related to each other and then using those relationships to make predictions about what's going to happen next.

All right. I love that all the components of AI technology are kind of mimicking bits of how humans reason about things and how humans learn. So we're sort of gradually getting all the components together.

I'm probably biased by having a very neurocentric view. That's certainly like my dominant scaffold for thinking about things. There's certainly differences too, but I think there's certainly some basis for interesting and compelling metaphors in that case.

Are there any bits that we're missing, like, what do we have in human learning that we don't have on the AI side yet?

I guess these aren't things that are fundamentally off the radar of folks, but I think they're still pretty active research areas and have a lot of open questions in them. One is this problem of generalizing outside of the data you've seen before. Basically, this is related to reasoning. It's also just related to innovation and creativity. Can you go from some knowledge you've learned and generate something that's. That's new? Can you make a novel insight? Can you propose an experiment? Or can you tie things together that haven't been tied together before? Or can you generalize even just to tell a longer story than you've ever seen in your database before? That this ability to go beyond what you've already seen, to take the parts you've learned and compose them into something new, either to constitute a novel conclusion or to constitute just a novel hypothesis about how the world works? I think this is something that is still an open problem. And there's been a lot of work trying to think about how to generalize to longer contexts. In particularly, this is sort of the metaphorical equivalent of tell a story longer than the one you've ever seen before. Recall something that's a longer time ago than you've ever seen in your training data before. These kind of problems are really starting to touch on these questions. I've done a lot of work with graph neural networks, which are a neural network system that implements something like relational reasoning. They're neural networks that learn relations between things, and then they're able to learn about novel or larger relations than they've ever seen before. Exactly. Because they're forced to break problems into parts that's like hard coded into them that they have to break problems into parts. And so I think this is a very key ability to want to be able to have in our most exciting AI systems. Another thing I think of a lot, partly because my research focuses on hippocampus, is just the role of retrieval and retrieval augmented generation. I'm super excited about that area. In general, I think it has a big relationship to extending context. It's sort of like if you extend the context of a language model, that means you can put more information into it. Similarly, if you augment the model with a retrieval database, then the language model can refer to things in this retrieval database that it hasn't necessarily seen before. And this, this just seems like it has enormous potential to expand the computational abilities of these models. There's been tons of work in the last few years making models that are really, really big and train on tons and tons of data. And the ability to now update these in flexible ways, use them for customized applications, update them when news changes. If you can kind of feed information into them in a more rapid and immediate way and still have them use it flexibly, I think that would make really good use out of the big expensive models we've made, and also just enable tons of different applications. Also just huge parallel to how we think memory in the brain works. So from the perspective of a nice batch of models to constitute hypotheses about memory, I'm really excited about that line of work.

So the idea of customizability just seems incredibly important. Like, no one wants to have to build Gemini from scratch. You just want to be able to customize it for your own use case.

Yeah, I think it's big. I mean, it's also just really important for factuality. I think oftentimes you want a model that's going to be able to tell you something that's statistically likely. Maybe it's never seen things before, but you've asked it a question it doesn't really know the answer to, and it gives you something statistically plausible and that thing is maybe true or maybe not true. And you know, they're getting better at making things true, but they're not always going to be true. A method for doing this better is to go into a retrieval database that has original sources. You can really, like, control what's in there and control what you're conditioning on and retrieve specific factual information that you have some ability to verify or think about its authenticity. And you can use that to Inform responses. I think that's also just a really important utility of this, is you're not just kind of confabulating. You can base this on something that you can evaluate.

So much exciting stuff going on. I know you've talked about some things you're excited about. Ready. Like retrieval. What's your number one thing that you're most excited about for the future of AI and neuroscience?

Yeah, I mean, I really like. I really like the memory stuff. One of my colleagues, Kevin Miller, he's used the phrase in conversations. Humanist neuroscience or humanist AI driven science, where the idea that AI could be used to give us new insights and help us learn qualitatively new things about the scientific world. I think that this is something that is a very exciting challenge for me. I don't know if it's something that. It's not something I'm excited about in the sense that this exists and we already know how to do it, but I'm excited about it as a challenge. A lot of the advances in AI for science have been to contribute tools to make a model that's really good at predicting how proteins will fold. It doesn't necessarily tell you why they fold that way, but it's a tool to help scientists do something else with biology. Now that they know this, then they can use it to solve other problems, or then they can try and unpack the reasons why it folds in a particular way. But I really like the challenge of let's not just have a model that's good at predicting, have a model that's good at predicting, and also can tell us something that we would consider an insight, which is challenging for many reasons, not the least of which because we don't have a very formal operationalization of what constitutes an insight, but we kind of know it when we see it. I really like Kevin's formulation of this. I think it's a nice way of thinking about what the next step of using AI for science will be.

Yeah, certainly in the world of data, that last mile, going from predictions to having some kind of insight that you can actually act on, make a decision about, that's always the hardest bit. So having some AI to help for that, and especially in the world of science, that just seem incredibly important.

Yeah. It relates to this question of what constitutes a good hypothesis versus what is just a hypothesis that you can feed into the scientific method. A hypothesis is just anything that makes a testable prediction. And you could have the wonkiest, weirdest model. You could just say this random Rube Goldberg machine I hypothesize is going to do the same thing as the brain. And that's an easy hypothesis to reject, and probably not that most people would be like, that was a kind of a boring hypothesis to make. But making predictive models that make predictions that you can test but don't necessarily decompose the model space into interpretable or understandable or semantic divisions just isn't usually what we think of as making an insight. It doesn't satisfy what folks are often looking for. So I think being able to do that extra step will be something very cool.

Absolutely. It's certainly exciting times. Thank you for coming on the show.

Yeah. Thank you so much for having me. It was really lovely chatting.