AI and Cognition with Daniel Sternberg

Welcome to Cognition. I'm your host, Rolf Nelson.

And I'm Joe Hardy. On this episode, we have a returning champion. Dr. Daniel Sternberg is with us once again. Uh, we were just reminiscing that the last time he was on the show was about a year ago. Actually, a lot has happened in AI since then, so we wanted to bring Daniel back to to talk a little bit more about artificial intelligence and particularly the relationship between artificial intelligence and neuroscience. What is neuroscience teaching us that can be helpful as we develop artificial intelligence and artificial intelligence systems? And then vice versa, what influence is artificial intelligence having on the research and science that goes into understanding how the brain works? So, uh, Dr. Sternberg is the head of data at Notion and has been working in, uh, data science for a number of years now since he received his PhD at Stanford in cognitive psychology. And, uh, yeah, so Rolf and Daniel and I have had the chance to work together in the past at Lumosity for a time and, uh, have stayed in touch and yeah, Great to have you back on the show.

Yeah, thanks for having me back. It's great to be back.

Absolutely. Yeah. So starting off with the sort of, you know, the topic of, you know, what can, uh, what is the sort of the mutual influence of neuroscience and AI? It might be useful to start by thinking about like, why is that even an interesting topic to talk about? Like why Why is that something that we would think about?

Yeah, I think like if we go back to actually our last conversation a year ago, we talked quite a bit actually about the kind of lot of like rewinding, you know, 50, 60 years to when neural networks, the kind of basis for the all of the models that we're seeing out in the world today, large language models, etc., their root going further back. Really go back to some things people were trying in the '50s, '40s, and actually '40s in some cases. And, and so like this interplay and relationship has actually existed for a long time and it's had ups and downs. But, but it's interesting to see, you know, now I'm curious if, you know, if you go to talk to AI folks, I think some people would say that we're kind of like trying to surpass the brain. We're trying to surpass human intelligence. It's kind of like take the best of what machines are really good at. You know, they can write— this can write code really well and they can run code, for example. And the best of kind of what looks more like human-like intelligence, we're going to talk a lot more about today. And so I'm curious, like, how you all think about this too, as we go through this conversation, because I'm not sure that everyone on the AI side of the world is just pushing the technology forward. And on the neuroscience side of the world, necessarily have the same perspective on what the goals are either. So we might end up talking about that a little bit too.

No, absolutely. I mean, when I was in, um, when I graduated college and was looking to go to graduate school in psychology and/or neuroscience, my stated goal at the time was I wanted to develop a system that would basically model the brain. In other words, you could have a brain that was in silica. So with the sort of mutual benefit of having a really smart computer and then also having a way to study our— fundamentally ourselves as human beings, like to use the artificial brain as a way to understand better our own experience and our own brains. So that's— that was like my perspective coming from a psychology background and interested in human beings. But to your point, someone who's coming at it from the perspective of wanting to solve a specific problem in computer science might not care, doesn't fundamentally care that there's any relationship between the brain and how their computer system works. So I think, you know, there's a few different ways to take that. I think we can explore these different elements. But one of the things to consider is, you know, in the current state of the art, how much is— how much are these new systems that are really exciting, like the large language models, some of these image recognition models, how similar or different are they to what we understand about how the brain works right now?

Yeah, and this is where we have— so we were talking before about how we have 3 cognitive psychologists here, and it's not typical that 3 cognitive psychologists get together and talk about this kind stuff. But I think we probably all think about this as using methods from cognitive psychology to kind of probe the mind of an artificial intelligence. And I know that's one thing we'll be getting today. So, so, you know, and it's not straightforward or necessarily easy to figure out what's going on in the mind. And, you know, there are methods that have been developed over a long time. I think it's really interesting that we're to the stage where we can now treat a model, you know, in a way that's like a human mind, and we can begin to sort of unravel, you know, to what extent does it have the same kinds of biases, shortcuts, kind of heuristics that normal, that regular people do, you know, does this all fall out from the data that's being collected for these models.

So yeah, so that's certainly one thing that I'm really interested Yeah, I mean, you know, thinking about, say, for example, taking specifically the large language model, you know, how similar— I think it's useful to think about both from the behavioral perspective of how similar or different are these models to how they behave, but then also underlying way that the computations are happening, how similar is that to something that's happening in the brain? Versus something that's very different from what's happening in the brain.

So there are aspects of the— of these models that I think if you zoom out a little bit, um, you can think of them as being similar in certain ways to the brain. So for example, um, you know, the way that they learn from input in a statistical manner matches on some level the way that groups of neurons and neuroplasticity happens, that synaptic plasticity, et cetera. But if you drill down into the details, the models will take shortcuts that brains can't take. So I mean, in an article I was reading recently, You're talking quite a bit about how, um, as I've known, as we learned, I learned in grad school, you know, backpropagation is, uh, is, is a core part of the how these models work, um, backpropagating error signals so they learn from, um, how close their output matches to expected output in some way. So for example, predicting the next word or something like that, um, or the next token in the cases of, um, the current generation of LLMs, which are, you know, chunk either like a letter or chunks of letters. But, but the way that they do that by backpropagating that error through many, many, many layers of a system is not really exactly how a brain works. And so what you see is in, in more computational neuroscience, neuroscientists actually, you know, decades now have worked on different types of models that can can do something like error propagation in a way that is more biologically plausible than the feedback signal flowing through the systems in the way that an error signal is in a model. And I think there's just a question sometimes to me of also like, to what extent is, you know, at what, at what level do we want to model a brain? Do we want to model a brain by literally recreating all of the connections between neurons in a brain, making sure that all of the— we model all of the biological processes going on, etc. Well, obviously, like, someone doing AI is not interested in solving those problems necessarily, unless, unless in some way it enables them to do something they couldn't otherwise do. But nonetheless, I think, I think a similarity between the models and how a brain works, in my opinion, is that you are at some level taking in basic input in some modality and trying to, trying to use it in some error-correcting way, which we know the brain does do. The brain does have systems and systems that involve error correction in some way to enable learning, whether that be cortical, whether that be, you know, reward processes, etc. These, these all exist. These all exist in brains. And at some level, we're —there are analogous problems to be solved from input to output and prediction in AI systems, but the exact details of how they go about doing that vary, and vary from the brain.

Yeah, but it is interesting how much in neuroscience, in the nervous system broadly, but in the brain in particular, feedback is really important. So that was one of the things that I found surprising learning about the visual system is You know, if you look at the signals going from the retina to the thalamus, you know, most of that signaling is actually happening in a feedback direction. Like the vast majority of that signal is actually not coming from the eye into the brain, but actually from the brain back into the eye, which is surprising. You know, it just— it shows how important that role of feedback is in neural processing as well.

Although I think the, like, the kind of feedback that you're talking about with backpropagation would be a little different than that sort of corrective feedback, right? So, I mean—

Well, that's a question, yeah. I don't really know how different or similar it is.

Well, maybe I may have just a really rudimentary understanding of it, but I'm thinking of just like, you know, something simple like a classifier where, you know, you're trying to determine if something's a tank or a car, right? So, you know, with that backpropagation signal, you're sort of telling the ground truth whether that picture actually does represent a car or a tank. And so you have accessible to the system something that humans don't necessarily have access to, which is sort of what the ground truth is, right? So that, I mean, that's what I think of as sort of a difficulty with that kind of backpropagation.

I mean, there are, um, so there are a number of, um, attempts to create something that kind of works like backpropagation but is more gnarly plausible. Um, and, uh, supposedly, um, while there is no evidence that backprop itself is possible in a brain, um, there are like different types of approximations that there is some evidence may exist in the brain. Though I, you know, I'm not an expert in mapping that back to a map, map back to brain processes specifically. I think a different way of like a different direction we could go here is to, you know, there's the— is the algorithm— is the mechanism the same as the mechanism in the brain? Separately, there's also, are the representations that models get similar to representations that we observe? Yeah, that's a good— even if the, the exact nature of the algorithm to get there looks different. And that's been something that's, you know, even before this current wave of large language models and vision models, et cetera, of this current generation, there have been studies for actually decades kind of showing how you can, through some sort of statistical learning model, end up building representations that look similar to representations in visual cortex, for example. So there's a bunch of research from the I would say like late '90s, early 2000s, looking at just basically learning the statistics of natural scenes and how you actually end up building out representations that are similar to the representations in primary visual cortex, et cetera. And just noting this idea of like, you know, the point being, you know, almost, you know, many statistical learning algorithms with certain characteristics will end up with representations and structuring the world in a way, layer by layer, that looks similar to what you get in— what you actually see in a brain. Which speaks a little bit, I think, to the— one of the things I find interesting about looking at these models, and similar to, I think, things we might talk about later in this conversation around language models as well, where there's sort of just a— you know, put the, put the exact biological plausibility of the mechanism aside for a moment. You can learn a lot about the statistics of input that humans likely receive through over the course of development just from taking a model that's really good at pulling out statistical information and assuming that the brain has various ways of doing that as well. So obviously the input that we get looks a little bit different than, you know, just being trained on the text from the internet or something like that.

Yeah, no, absolutely. And I mean, I think that speaks to the point of, you know, as we think about similarities and differences in artificial versus, we say, call natural intelligence or, or human intelligence, um, one of the key things that drives them in the same direction is they're operating on the same world. But, you know, the, for the most part when we are working with artificial intelligence systems, we're interested in solving human-scale problems. We're interested in doing things that we find interesting and useful. That's just because we're human beings and we care about things that human beings care about. So we make the artificial intelligence care about things that human beings care about. So we train them on inputs that are the kinds of inputs that human beings would use as well. Things that are visual inputs to a human eye, things that are auditory inputs to a human ear, words, you know, these kinds of things that are human-scale inputs from the world that we all live in. So it, you know, just if you think about homologous, you know, structures or frameworks within these systems, it would make sense that there would be some similarities because they're operating on similar statistics in the natural environment. And I think in the large language model, this sort of speaks to, you know, the next layer up is in a large language model, for example, which has been trained on the corpus of, say, all of the internet. It has the statistics of the internet and its sort of worldview, if you will. And, you know, it tells you a lot about what's out there on the internet as we talk about like biases and things like that. What I find an interesting question here is, to what extent does, based on the way that the large language model work, does it actually have a representation or a worldview based on how it, you know, that's somehow embedded in its structure? I mean, because a lot of the feedback systems that the way that the brain works is that has sort of like a canonical view of what should be there, makes predictions. About what should be there and then is working off an error signal of how different the thing in the world is from that. You know, it's got kind of like a worldview or understanding of the world embedded in it in that sense. And to what extent does the large language model, for example, have a worldview embedded in it?

I think, I feel like I might be repeating myself from a year ago, but I think— That's okay. That's okay. I think it has more of one than we might have thought. Like, I'm not— I find myself kind of in the middle of this debate, which I find myself in. I think it has— like, it is surprising the extent to which just training on text can, can allow you to, like, build up more complex representations. And I think some of the things that the models are able to do, yes, it is quote unquote fancy autocomplete, That's the kind of skeptical take on the whole thing. But, but to get as fancy as it is, I think there are some representations that you need to build up over time. However, I don't think it's enough. And I don't know the extent, you know, the extent to which it's the multimodal models that are now being trained in a lot of cases. So a lot of the really recent models that we've seen come out post-GPT-4 have been actually multimodal models, or some version of them is multimodal, which means that they have input like video input with text attached to it. They have, you know, image, audio, all like many modalities simultaneously being trained, but which may help some. But they're not like a robot that's walking around in the world and getting input from a bunch of sensors. But I think it is more, it's been surprising that it picks up on more things than that are, we would think of as being complex and requiring some level of world knowledge than we might have thought was possible with just these models alone. On the other hand, I do think the folks who argue that, yes, these models seem quite powerful, But there are additional kind of systems and processes that we are not— that we have not built yet. Like, we haven't really created separate systems for planning, for example, like planning and action. And so like even planning and action are still dependent on just like iterative, kind of like a recursive loop within these models, within like asking these models to like do something, take an action, put prompt themselves to do something else, etc. But they're still— it's all still just happening with this one large language model, um, that is trying to pick highly probable next outputs. Um, so I do think, like, I don't know that we're going to hit a wall, but I, I personally— this is very much personal, uh, not where I work or anything like that— like, I personally think that, um, there are other there are other components of a full kind of artificial intelligence system that we will need in order to really achieve like what we'd look at, look at as like human-like intelligence in a number of cases. And so it'll be interesting to see as we especially start to work more on things like more autonomous agent-like behavior in these models, is the kind of LLM plus prompting plus a couple of other tools that has access to enough.

Right. Yeah, I think—

go ahead. Oh, I wanted to drill into the idea of representations because I think that's a really interesting— you know, like you say, if you think of it as just a glorified autocomplete, it's hard to imagine that there is a real representation inside an LLM. But then again, it's not really easy or obvious to see how we have representations instantiated in our brains too. There's, you know, you can— neuroscientists don't, you know, you can't just find a representation in there by cracking open the skull and looking. How do you go about looking? I mean, again, in cognitive psychology, we have ways of sort of indirectly testing, you know, and you have some intuitions about this too. So I guess that comes with just interacting with the model, but how do you sort of figure out the parameters of what a representation might be for a an AI?

Yeah, I mean, in, in, in an AI, um, how do we look at representations? I mean, historically, um, you would look at statistics of the rep— you know, provide an input, look at the representations in internal layers of that model, um, and look at them statistically, essentially. Uh, you're looking at the weights on different Well, you look at the weights, you're also just looking at like, if I give you a particular set of inputs, um, what is the activation in one of those hidden layers look like, um, at different points, um, along, you know, if you're having something that's like a recurrent— well, back in the day, this was me aging myself, uh, in like, we don't use— they don't use recurrent neural networks anymore. They use, uh, these transformer models now, which don't have to have recurrence and have all these nice properties, but Back when I was doing neural networks in the dark ages, the dark ages of the early 2000s in my case. It was like the AI winter. Yeah, exactly. It was all about— this is a real aside, so feel free to throw it out. But it was all about when I was in grad school, it was all about Bayesian nonparametric models and the Bayesians were really hot in the mid to late 2000s. And then in ML, in the ML world, it was like, I don't know, different types of models, which I won't go into right now. But back then, like, you know, you could do things like, you know, I would, if I were having a model that was trying to, you know, predict the next word, I would just look at in this like hidden layer or context layer, what was that representation like? So, you know, there's a model from the, this is a very simple model from the early '90s called the Simple Recurrent Network, which is Jeff Elman's model from Jeff Elman, who's a cognitive scientist at UCSD. It was a really simple model. You basically had a feedforward neural network with normal input layer, hidden layer, output. And all you would do is essentially feed back the prior hidden layer activation as another set of inputs so that it had some prior context from that. The problem with it is it like loses context really rapidly. Like that, that is not enough memory essentially to do a good job. And, you know, models, many models since that time have built up way better ways of holding on to context over long periods of time. And Transformers are kind of the latest iteration on that. But, but yeah, you would look at like literally at different points in a sentence, what was the representation in that? In that hidden layer to kind of see like what did it know about the sentence. And you could do like hidden layer activation comparisons. You could do principal components analysis. You could do all kinds of like statistical analyses to try to like build up a story about how they work. There's much fancier ones now. So, right.

So on a smaller model, I mean, it would seem like you could, you could get a handle on things, but you know, when you've got like 175 billion parameters or something. Is that something you can make any sense out of under the hood?

You know, I, I have not actually tried to do this recently. I would say like what I see people doing more often, I mean, there are some like fancy statistical techniques and I've seen some very cool visualizations and stuff like that, and I forget exactly how they're built. But the other thing I've seen like more commonly with these types of models is try to get at some idea of what the representations might be by, by, by, by looking at what predictions they make, for example, in a similar vein. So I'll go maybe down a rabbit hole for a moment. One of the like big— maybe in the last year, one of the big things that's been happening in AI world in terms of like applications of these models has been this thing called RAG. You know what RAG is? No. So RAG is Retrieval Augmented Generation. So essentially all of these, you know, a lot of product features. We have one at Notion, which is like our Q&A feature, which we launched last year, late last year in beta, is, you know, you take a you take a large language model and then you give it access to a way to retrieve content, like basically do a search of content from somewhere. Or another product I really like called Perplexity, which is a— it's a web search, LLM-driven, like web search product that's like for knowledge. It's really, really cool. It works really well. And you're giving these models a way to search. And then take some content back. Right. The basic idea— I won't give away any secret sauce, but the basic idea, you can look this up anywhere. RAG is the acronym. RAG, you're, you know, basically say, hey, model, like you ask it a question, it constructs a query itself. It, it searches some search system, pulls, pulls some end pieces of relevant content load and then says, using this content, answer the question, um, or, you know, provide an answer, a response to this.

Okay, I think I know what you're— I think I know what you're talking about now. I think I've heard of this.

Yeah. And so like commonly, one of the common ways that that search happens is through some type of semantic retrieval. And so add in the idea now that actually what people are using is what's called like a vector database, where I have taken all of this content, chunked it up in some way, and then I have fed those— I, I have a, I have an embeddings model. Like, it's a basically, it's a large language model. What you're giving me is some internal layer of that model. So, uh, I provide a big chunk of text and it provides me back just a vector, a bunch of numbers, um, from an internal layer of that model. And, um, or how they may do something fancier than that and combine multiple layers in some way. But essentially, and then when you are doing a search, you actually retrieve that way. So you like construct a new query and then you find the chunk, the, the text that is a closest match in this vector space to that query. Um, and so there again, like, you can start to see how the— these vectors are representations from the model. Um, and again, like, I don't know how easy it is to like, you know, visualize all of this, or the ways people are doing that these days. I haven't been as close to that because I'm not kind of deep in that part of the research. But, um, you can certainly start to see like, what does it find similar, for example, based on, um, based on like some set of text. And so that tells you something about what the model has sort of learned about, learned about the world.

Yeah, I mean, in many ways that kind of, uh, Representation, looking at those representations based on the response to a certain stimulus input is a lot like mapping a receptive field in a brain in certain ways. Like what are things responding to and what does that tell you about the role that that particular part of the brain is playing, for example. And in this way, it kind of you could imagine probing the neural network in much the way that a neuroscientist probes the brain.

Yeah, I think an interesting thing too is like in a lot of these cases where you mentioned the receptive field of neurons, I mean, this may be an aside, but we have a much easier time understanding the kind of representational properties of non-human mammal brains than our own brains because we can stick electrodes in them. So, so I would actually say, you know, reading through some of the articles we were talking about in advance of this conversation, It was interesting to me, it was intriguing to me thinking about it as like, well, actually in most of these cases what we're really talking about is, oh, this recurrent neural network, you know, this aspect of it has similar properties to, you know, some primate neurons in that we have that we— Yeah, some non-human primate. Exactly. Yeah.

Well, I mean, that gets to a good question, which is, you know, you mentioned earlier that, you know, what we're really kind of getting excited about interested in is that a lot of times these networks are becoming as good as or better than humans at things. And I think that this to me is sort of like prompts the question of like, when is AGI coming? And then of course you have to answer the question first, which is what is AGI? What is artificial general intelligence? And I think that I actually know the answer now, which is an improvement over the last year.

Um, tell me.

So it's, it's actually very, very simple. Um, which is that, I mean, so first you have to define intelligence and I certainly know the answer to that now, which is, as I've been saying for a long time, but now I'm certain of it, it's just how people think. How human beings think, period, end of story. Like, we say that someone is intelligent to the extent that they think like us, or that they solve problems that we find interesting better than us. We would call them more intelligent than us if they're solving questions that we particularly find interesting and important in ways that, you know, that we may not be able to, uh, critically as human beings. We fundamentally understand that someone is not intelligent or not intelligent. There's a range of intelligence and that there's some sort of, some sense that there's some multidimensionality to intelligence. But nevertheless, you could say that someone is more or less intelligent. So it's not that machines are going to— so if you think then about artificial general intelligence, you will have artificial general intelligence when the machine can solve most or all of the problems that humans can do at least as well as human beings. And so the point is, actually, I think we have artificial general intelligence already. It's just not especially intelligent. So if you think about actually artificial general intelligence should also exist on a continuum, all kinds of different systems are solving problems better than humans already. And they're able to solve them flexibly somewhat. In other words, they're not just solving one tiny problem. It's not like a calculator that's just solving long division, but can solve a variety of problems somewhat flexibly. Like large language models certainly fit into that category. So there is some level of generality and there's some level of intelligence. I think we already have artificial general intelligence. It's just not as intelligent as we are across the full breadth of things that we're interested in solving. Interesting.

I, um, it is also interesting to think about this in the context of like, we also as humans, uh, and as, you know, psychology, history, in the history of psychology, we decided that certain capabilities of humans kind of fell under like what counts as intelligence versus not. And one could argue that even decades ago with symbolic systems entirely, it wasn't human-like, like you were talking about before. The things they were really good at were things actually we found really hard. But you want to do factoring of extremely large numbers or things like that, they could do them really fast, really efficiently, much better than a human could ever do. And that was decades and decades ago, literally the first computers. Probably could do things that a human couldn't do very easily. That's why we built them. But, and why they caught on. But, you know, even if we look at what humans can do and what machines can do, you know, in some sense, I think, you know, 10 years ago, you would have said there was like a famous, like, you know, XKCD, the like comic, there was a famous example of like, there was, I forget what it was, it was like, I can pull it up. But there was some, the joke has to do with like GPS, is really easy for a computer to do. Like if I want to geotag the location of this photo, it's really easy. But if you want to tell me whether there's a bird in that photo, it's going to take a research team and 10 years or something like that. I'm butchering the joke, I'm sure. And, um, and like now actually, like we basically solved that problem. Like we can find a bird in a photo really easily. Um, and like anyone can grab some API off the shelf and it'll do it for them. And it's not expensive even. But so, so that's something that it turns out was like really hard for computers for a bit. And then we just figured out a better algorithm and the algorithms got better and then it was actually not that hard for computers to do it with the right amount of input and all of those sorts of things. But like there are other things that you like and those are sort of things that maybe fall under, well, I would need that to be able to show AGI, right? To be able to show intelligence. Well, I need to be able to recognize objects and things like that because if I want to like do an intelligence test or something like that, these are some of the basic skills you would need to have. On the other hand, like, you know, the robotics side is like still pretty early. And actually, when I was talking to someone the other day about this, who's been following kind of the how these new AI models are affecting robotics and driving it forward, and it was basically like actually like they've solved a lot of the kind of basic planning and reasoning, but the actual like action stuff, like with like, you know, sense and like feedback loops with with like sensory and motor processes is like still really hard.

But we don't think about staying on balance, like even just being able to take steps forward, staying balanced is like really surprisingly challenging.

It's strange to think that folding clothes would be something that takes— it seems like it takes a lot of intelligence.

And I would buy that thing like in 2 seconds if there were one that was good. But, uh, but yeah, it's, uh, that's really hard still. I don't know if that's gonna just fall under the We don't have the right, the right training data to the right degree yet. Plus we don't have the right exact algorithms and we'll just find them and then solve the problem. But it's interesting to me because like we could get to AGI and like never have that, even though like, again, if we really want to like say like, does it do the things that humans can do? Well, those are some of the most basic things. Like my kids learn to do that long before they learn to do all of these other things. Yet the machines can't do it right. My 2-year-old can pick up a cup better than a robot can.

Yeah, and that's all that stuff that just seemed like it would be really straightforward and not difficult at all. And it's not, you know, you don't pay people a lot to do those tasks. You pay people a lot to figure out, you know, financial kinds of things. And it turns out that stuff's pretty straightforward.

Yeah, that's a really interesting point. Yeah, I think the idea of like there's the disruption of these models on society and the economy is actually like coming for like certain kind of mid-level knowledge work, right?

Yeah, yeah, yeah. It's like if you had made a bet 20 years ago, who's going to be out of a job first from AI and robotics, like a mail carrier or a lawyer, right? You probably would not have said the mail carrier. That, you know, you probably would have said the mail carrier is going to be out first and the lawyer is going to have a job for a long time. But actually, that certainly looks the opposite.

Now. Yeah, it's really interesting. I mean, I would say like my, my view on like what are, what that as someone like kind of working with the products that are built on top of these models, what are, what are the capabilities right now that like people are anywhere near actually shipping or are shipping in the world? It feels a lot like it's like the equivalent to hiring a contractor or an intern, maybe. But I would say better than an intern actually, because it can get there faster than an intern. But it is a little bit like a contractor. It's like, I need to spell out for you exactly what I want you to do. And the exact criteria and then have you go and do it. You can't like, uh, expect it to like think outside the box and, and come up with the problem, like the solution itself quite yet. You need to like specify things in a very clear way. And it's like, I'm handing it off to you and I'm gonna go check on it when you come back to me with your results next week or whatever.

Yeah, I mean, this is where like, if I, you know, today if I had to guess what the future of say software development was going to look like in 5 to 10 years, It would be that everyone's job is going to be a lot more like a product manager and a lot less like a developer than they were 10 years ago, or certainly 15, 20 years ago. And you know, it's like you have to be able to specify what you're trying to achieve very clearly and it needs to be the right goal, but the actual development of the software is becoming increasingly easier. Yeah.

It's interesting to see how even in just in the last year, I feel like the norms in software development have. Shifted, at least where I work, like I hear much more, many more people saying like they are using constantly and honestly becoming somewhat dependent on, uh, dependent on like Copilots for, um, for, for writing software. Um, it's become quite, I think, common at startups to, um, for engineers to have adopted these pretty heavily. Not everyone. It's like there's a gradation of people who are and aren't using them more heavily. But to the point that, you know, I've seen getting feedback for other tools we have at the company where they're asking these, these other, these other vendors we work with, hey, you know, I'm used to working in this way, but when I use your tool, you don't have as like you don't have a Copilot-style autocomplete. And I, I'm really very used to having that now. Is there any way you're going to add that in the very near future? So it's definitely, it's definitely already shifting things quite a bit.

So here's a, here's an aside too. So what do you think about kids coming up and learning coding. What do you think? It's in sort of flux now because it used to be like, okay, coding is going to guarantee you a good, well-paying job. Now, not so much. It's unclear. I mean, there are a lot of people who go through that and don't necessarily find a great job doing it. Maybe there's something useful about still learning the logic of it. What do you— what's your— what's your take on that?

I mean, it's, there's like a short, medium, long term. If I were a teenager, like I would probably still say there's value in learning that way of thinking. And that again, like you're, while you don't need to produce all of the code yourself anymore, like at the end of the day, any company, you know, the company I work at is, I don't know, between 600 and 700 people now. And it's like a couple, you know, 200 engineers or so. And with 200 engineers working across codebases, like whether you're like having AI do a bunch of it for you or not, you're going to have to think about all of the side effects that it has on other parts of the codebase system, etc. And it can probably like help you do that, but you certainly can't trust it to like, to like think, think through all of those implications for you.

You'll always need a human. You'd always need a person, a human at the sort of the top level, at least.

At least. And so like, yeah. And so I think like if you're kind of, you know, you're in college, you're a computer science major or something like that, like, yeah, like your career is going to look a lot different than you probably thought it was going to. You might have thought initially it was going to, or we would have thought 10 years ago. But I think that's relatively safe in the near term. I do think the question of like, I have young children and like what, what actually is valuable for them to learn in that area right now is really hard to say. My basic assumption is being good at like math and computational thinking is still a valuable thing. And the people who are building the systems also need those things. So, you know, you can always, you know, end up, end up actually working on these tools themselves.

I mean, when I was, when I was in college, right, you know, computer science was still like part of engineering fundamentally. I don't know if that still is the case. Seems like it's less and less like that, right? And it's more like there was a period, I think, but I think it's gonna have been a fi— we'll look back in historically and say this is like a finite period of time where like learning Java was like a career building, you know, something that would really guarantee you to have a good job as in, in your career. I think knowing Java or knowing C or knowing C++ is no longer like a primary job qualification. You have to understand What you're building, you have to think about, you know, systems. So I think it's like, it's back to like learning engineering as like a broader subject matter, you know, particularly as you think about building the systems we're talking about in the future, which are going to involve more robotics, they're going to involve more effector systems, sensor systems, and how those interact. So you're going to need to know a lot more, I think, just thinking systems-wise from an engineering perspective is going to be super valuable, as well as like the more product management perspective of like understanding on a human level what is the job that's being done by this technology and what's the customer hiring it to do.

For sure. I mean, one of the things that we run into now is, you know, for example, on the AI team that's part of my group, we kind of have two sets of engineers on it. We have kind of more like More what you would think of as kind of similar to your like machine learning engineers from, you know, that we've had. I've hired that type of role for a number of years now, but though it looks a little bit different in terms of like what they actually end up doing because we're not building foundation models, we're sometimes fine-tuning existing models, mostly using APIs, things like that. But still a lot of like how do you think about like measuring success of the model, like evaluation frameworks, those sorts of things for the specific tasks you're trying to like enable the model to do. But then we have this like kind of other role, which is like an AI product engineer role. So it's like a more— in some sense it's like being a regular product engineer, but it's something I think that this is the one where I'm like, I don't know that I say this, I'd be careful what I say, but I was like, I don't know that this is a role long-term that is like differentiated from the rest of software engineering. I think most people agree with this. I think everybody is kind of ramping up on how to like do software development for AI, like AI-driven features. And it looks a little bit different. It's not like you don't need to be like a machine learning expert. You don't need to like go deep on the math or anything like that. But you're like, you're like doing software development of features that involve non-deterministic systems. And that's like very different, I think, than what a lot of product engineers are used to. And so getting used to like how you prototype and like build for that is, is just different. And I'm sure they also use a lot of these Copilot tools and things like that along the way. But it's also, it's just a different flavor of engineering, but it's very trainable, I think. For what it's worth, like for us, like we found actually like younger, more junior, honestly, and engineers in a lot of cases are better suited to it or get there faster. Because I don't know, maybe they're growing up with these technologies coming up as they're like, you know, in college and, and also just like they're not as kind of set in their ways about how software engineering works.

It seems like, I mean, in the workforce, people are going to be forced to be more and more human, right? Yes. Right. Like you're forced to not do all of those rote things because that's— somebody else can always do those things. Is that going to be fun or is that just going to be harder and harder?

Well, I'm going to— this is again an aside that you can feel free to use or not use, but I think about this in the context of my kids actually a lot. Yeah. And, and like, you know, I am who I am, so I still care about like, oh, they need to learn like a good level of STEM curriculum and all of those sorts of things because these are things I care about personally, though again, I don't know like which parts of it are going to be most valuable. Valuable or not. But the other thing is like we've leaned in, in like educational decisions for our, for at least our older one toward like, let's make sure they like know how to resolve conflict with other humans. They kind of know, yeah, they, you know, they know how to work collaboratively because these are all things that probably we will need to do, whether it's fighting our AI overlords, just right, right. Or, or just, you know, to operate in society. Like we still need humans to make the decisions for the human society.

Society. Yeah, exactly. Speak another language. All different ways to get along, right? Different ways for, for human beings to get along. Because fundamentally, that's— as we think about the problems that we're solving as people with technology and the problems that we're not solving with technology, uh, I mean, certainly we can do things that will help us grow food more efficiently. Hopefully we can solve things that help us get rid of some of the waste that we've created, you know, with technology, all that kind of stuff. But fundamentally, they're not— this technology is not helping us resolve our conflicts at all or get along any better.

Yeah, it may give us tactics, right? Like, you probably get some tactics for a given situation, but kind of knowing what— how to stack rank the importance of different problems we need to solve, etc., like, you need human values and judgments to make those decisions. Um, uh, because that's to be like what we care about, um, not what, uh, just, you know, a machine rationally thinks of it. So it's a At least that's my take. Um, I'm sure there are people out there who disagree with me and think the machines can figure it all out.

No, I, I 100% agree with you. And I think it all, it, to me, it, it all is of a piece with the definition of intelligence, which is like fundamentally we're asking these machines to solve human problems at a human scale cuz it's just what we care about. And ultimately what's gonna be most important for us as human beings is figuring out what is important and how can we surface that? How can we center that in all of our decision-making when it comes to technology and other, all the different ways in which we're engaging in development and commerce in our society?

So one of the things I wanted to make sure to get to talk about today, because you had sent us an interesting paper on this, on using AI as participants in research studies. And this is a really interesting idea. So, okay, obviously in research, you know, it's expensive and time-consuming to use participants. And I think anyone who's ever run a research study would be happy if there was a perfect AI that they could just do everything on, be quick, almost instantaneous, you come up with neat results, you could use a million subjects if you wanted to. But obviously, I don't know if we're quite there yet, but I mean, there's some, some, some positivity about the idea about, about this happening. And there have been some studies that have already used AI agents and replicated some kinds of psychology findings. Like, I found this fascinating that there was a study that in a sense replicated the Milgram experiments. So, you know, these experiments from the '50s of obedience to authority, where after a certain point, people will— well, people will continue to press the button and give you electric shocks for a long period, but after a certain point they won't. And they found that, you know, that AI would, I guess, under prompts, still behave in a similar sort of way. So I would love to get your take on where we are Are we— what does it mean when we're probing in a large language model on something like this and what can we get out of it?

Yeah, it's interesting. I— so going back to something Joe said earlier, wouldn't it be great if we just had like a brain in silica that we could like— well, if I know it's a 1-to-1 representation of the brain, then I can just provide it the same inputs and get the outputs and trust that they're like the right outputs. Of course, it has to be a specific brain. It can't be, you know, every brain is a little bit different. But, but, you know, I got a distribution of like replications of people's brains and I could do this and like that would be ideal because I would just know that, well, I know I've done this one-to-one mapping and it works just like their brain does. So I can, I can just use it in place of them and I can, you know, spin it up on a server somewhere. But, but in the absence of that, We kind of have to know. We, you know, we can test, we can test that a, a large language model, for example, shows some similar features in terms of the way it— I'll be careful about saying it reasons, but it does reasoning tasks. It responds to reasoning tasks that is similar to the way that humans that we have data from experiments that humans respond and some of the biases and ways that biases we have, etc. It would make sense that they have some of those biases because they're trained on the output of humans. But, but I'm actually— I have kind of two different thoughts on this. I think there it makes sense that there are biases that are similar to human biases given the training. So it feels like there are tasks in which we should be able to use it. We should be able to use these models. And there's, you know, on the one hand, studies to show that these models are showing similar biases and tasks like this paper we were, we were looking at around like context effects and reasoning, which you can go into more in a second. That, that tells me that there are some similarities in some way. I don't know that it's similarity in the mechanisms, but similarity at least in terms of the inputs and outputs. But would I be willing to extrapolate a novel finding without ever testing it on a human? Yeah, I, I don't know why I would be willing to do that. I think it would be very cool for generating quick, specific predictions about human reasoning. So like, given the historical Given the— what we know, for example, about human reasoning, I might be able to test some novel ideas, like a bunch of them, on an LLM, pick the ones that show some clear, some clear trend, and then go and test those on a human afterwards. So I might use it as a tool for quickly deciding hypotheses to pursue. But I would never— like, are we going to be willing to publish papers in psychology journals that are like, well, the model showed this, so humans show this? It's like, no, I can't extrapolate that.

That's right, because, yeah, I mean, what you're going for, what you care about, is how humans perform, not necessarily how the model performs. I mean, if you're— if what you're doing is really just trying to figure out what the mind of an AI is like, that's different, I guess.

But, but I mean, I think though, we're like taking it one step forward, if you had multiple brains and you had a whole bunch of them, and then so these are autonomous agents now, so these are like modeling a society. Now you could do some really interesting experiments that are just impossible to do ahead of time that might give you some help in making decisions, like for example, with policy. So we were talking about conflict resolution. Imagine you modeled two warring factions and, you know, the behaviors of those individuals, you know, uh, I guess it's sort of like an extension of, let's say, the AI game theoretic approach, right? Like, so like, what would that, you know, how would they respond to this type of intervention towards peacemaking, you know, or that other, you know, a different approach? How would that play out? Might be some interesting ways to, you know, generate hypotheses there that could be, you know, it'd be difficult to test. You just get, it's like always like a one-shot in the world of like how you, how you deal with these policy issues.

Yeah. I like that idea of, of using it for testing hypotheses or sort of as preliminary analysis, but not necessarily something you're making conclusions on. Yeah.

I mean, in the, in the paper that we were reading on, uh, using LLMs in place of human participants. Participants, did they actually— I'm trying to remember if they directly advocated for the idea of like just literally running studies where you never have a human. I think they waffle.

I think there was some appropriate waffling and sort of toning it down so that it wasn't like these are now humans that you can just report these results in your— you can do 30 experiments a day and publish them, right?

Yeah, I wonder, like, maybe there's an intermediate version here where, uh, I'm thinking back to my, um, to my time when I was actually writing papers. I don't really write papers anymore. And you know, you sometimes would see this thing of like, you would run a behavioral experiment, you would get, you know, you had some hypothesis, you would run some set of studies to try to fence in like what the potential interpretation, correct interpretation was. And then at the end you might have like, and we like built some model and it made these predictions and in a future paper, in a future study, you know, we're going to test these or something like that. Would we be comfortable with the idea of like you run an experiment, um, and then, uh, you, uh, on humans, um, you get some outputs, you replicate those outputs in an AI model, make some novel predictions, and then like come back later. So you have this like final section of the paper that, that looks at some of those predictions.

Yeah, yeah, yeah, yeah. So I, I have a bonus question for you guys that came up Uh, this morning as I was on a walk with a friend of mine, we were talking about, um, AI and intelligence and like, you know, the definition of intelligence center. He had a question that was a little different take on, on some of this than I've heard before, which is, would, for you to consider something to be like an AGI, like a real true artificial general intelligence, would it need to have some kind of self-awareness or self-consciousness? Certainly, you know, feels like you would need to at least have it have what we're talking about is like some sort of worldview or representation of the world. And I think also probably some sort of intention about what it was trying to accomplish. Now, maybe you could say that that intention could be programmed in,, you know, with the sort of the rules of robotics or whatever, of robot, you know, but like, it, it, I wasn't sure what the answer was to that. Like, would it, would it be necessarily, there'd have to be some kind of sense that this thing, uh, this AI knew what it was doing and why?

I think this goes back to, uh, what Daniel was saying earlier, um, talking about sort of kind of modules or capabilities that an artificial intelligence might have, and, you know, stuff like attention and memory. And I think one, one thing that you could talk about there is just a sense of self. I mean, we have, uh, you know, we have a specific system in our brains that let us know what ourselves— what, what's a part of ourselves and what isn't. And, you know, I guess we've talked about that before in with respect to psychedelics, where you can, you can experience no self. Um, so, you know, both experiences are possible, but our brain causes us to experience a self and, and, uh, you know, experience the contrast between ourself and other things. And that may be— and is that essential or absolutely necessary in, in artificial— in a generalized artificial intelligence? I could see it as being part of the package.

Well, this is— it kind of gets to the, the core of the question we started with, which is like, what is the relationship between neuroscience and AI, right? Like, this is an area where you might say, look at a human being that's evolved over all these millions of years, and it seems to be that this sense of self is like a core, core element of what it's like to be a human being, and it might be important for some reason. Like, you know, like nature has expended a tremendous amount of energy or whatever it is. There's a tremendous amount of energy associated with the sense of self and maybe it's important. Yeah.

Yeah. Without getting too deep into the philosophy, because I think there are like you have to have different people have different philosophical stances in general on what do we mean when we say self-awareness and— Yeah, certainly.

We, you know, we talked about that on the show quite a bit and Ralph and I have different views on that.

So— Oh, interesting. Okay. Okay. I'll try not to outmine right now, but I don't want to know that I disagree with either of you. But I, I think like some version of self-awareness and intention, I think it's useful to the extent that it's necessary. I don't— it's necessary for the model to work. I think, I think like it's a separate question to me just in the definitions I would use. Of like AGI does not— AGI only requires that if that was required to achieve behaviors that we would say correspond to AGI. I'm outing myself as a slight behaviorist right now, but which I kind of am and always have been. But like, I think to the extent that, you know, you need a representation of self or intention in order to behave in ways that we would think of as being intelligent, then maybe that is necessary. I think like there's very minimal versions of that that people do with LLMs today, right? Like a lot of these models, the ones that you get to use, um, directly in consumer products, tend to have been like prompted with a bunch of things about how— what they should do, how they should act. Um, you know, I am, uh, you know, I, I'm trying— I'm a helpful agent who's trying to help you do XYZ. Like, so this should inform the way that you act, essentially. And they are also obviously trained over time to give different types of outputs and be steered away from certain types of outputs that we think of as harmful, etc. So there's both like explicit and more implicit ways of training that. Um, I don't, I don't think that corresponds to like, um, world knowledge or like an actual sense of self-awareness necessarily. Um, but we do find those are useful for getting certain types of outputs we think think of as being human-like. So to the extent it's required, I think it's, um, I think it's important. I wouldn't be one to say, uh, that some— we need to know the model has some quote-unquote true self-awareness to qualify as AGI, because first off, I don't know how the hell you would ever measure that. And if it's not measurable, then what are you supposed to do?

Uh, what I like about it though, I agree. I mean, and that's why— but that's why I liked the particular phrasing, or like, so it was like a, just like more of like a, an appro— where, where he was coming from, right? Like, it's like, actually, it tells you something about what you think intelligence is, of how you answer that question, right? Irrespective of what you think, like, what, what you think about awareness is. Yeah. Which is, I think that's a harder question, right? Like, But it, so it kind of gives you a little bit of a lever into thinking about like, you know, what do we mean by artificial general intelligence? Like, would it be like if it was just a machine that just only did exactly what we asked it to do all the time? Um, would that count? Because actually it seems like intelligence to us in some way implies, uh, you know, working in the environment in such a way that benefits the agent. Yeah, yeah. Like, it is like there's some self-benefit to the self that if someone acts in ways that just like are— is like, seems like always counterproductive to that person's interest, you don't consider that person to be especially intelligent.

For example, again, I'll put my behaviorist hat back on. Uh, you, uh, you need to give me a way that I would like actually empirically measure that. First, at a minimum, for me to be able to say that it is required. So I need to know, like, we need to come up with— I'm sure a psychologist is going to do this at some point— like, how do I— measures of intention and in large language models or in AI systems. I'm sure probably someone has already worked on this. So you need, you need that. And then I think, and then I think to your point, you need to be able to say that have some way of thinking that, showing that this is required to perform in certain ways that we consider to be intelligent, for the model to perform in certain ways that we think are intelligent. So that seems plausible. The flip side version of this is we'll act like— I also think humans overestimate how much of the— what percentage of the actions we take are kind of at that high— operating at that highest level of self-awareness. Like, literally, like, I— maybe I'm outing myself and how I act in the world. I always worry that I'm doing that. It's like everyone else is actually like really thinking through every, every action they take very carefully, explicitly, and like through language and like, you know, internal monologue the whole time. But, you know, I think like, there's— I'm thinking as I'm speaking right now. But also like, my speech is slightly ahead of in my thinking in some ways, unless I'm being like very careful in giving a speech or something like that. And so I do think, I do think we overestimate that about ourselves in some ways as well. And a lot of what we're— I thought surprising about the performance of some of these models and makes people like Noam Chomsky say in the New York Times that they, you know, can't be doing what humans do, I think rests on a bunch of like non-empirical assumptions about how humans work and how most human action happens. And I do think there's, uh, people have very different opinions. You mentioned the two of you have different opinions about some of this. Like, people have very different intuitions about how, um, we operate in the world, and especially around things like degree of self-awareness, etc. I really felt this, um, when I was in grad school too, that like, I was like, my advisor has my one view of the world, and like there was entirely opposing views. And I was like, these people are convinced they are, they are operating in the world in like very different ways. And that's also quite interesting to me. And so like, I, but I think a lot of that is non-empirical. I think a lot of it is, is very hard to wrap your head around, uh, get kind of your grasp on just through empirical data. And I don't know, like, I, I don't know if I want to try to, uh, if we have to go there with our, with like building AI models, I think it's going to make our lives a lot harder. Um, well, it'll be good.

It'll be good. I mean, this is the thing where, you know, this— in this, uh, lawsuit between Sam Altman— well, it's between Elon Musk and OpenAI, but, uh, you know, where Elon is saying, look, you guys already have artificial general intelligence. Yeah. Why is he saying that? Because, well, actually, you made the decision that when you reach artificial general intelligence, you have to make sure everyone has access to it. So you're not going to be able to make as much money. So, you know, it's who has the incentive to say artificial general intelligence is this really big thing versus who has, you know, who's motivated to say that this, you know, artificial intelligence, artificial general intelligence is actually a subset of the things that we, we think of.

And now we have a judge who's going to have to make some decision on whether we've had AGI or not.

So there are stakes.

There are actual stakes. Yeah. And there's going to increasingly, I mean, you can imagine all the different stakes that become, of course, then you have the, you know, the, you know, the famous things from, you know, from all the, you know, novels and, you know, literature associated with, you know, robots and androids and the, you know, whether they have rights and so on and so forth. But yeah, there's a lot of, there are a lot of real practical policy implications of the stuff that we're talking about. Yeah.

As well. I just think, I just think like if for intelligence, I would be concerned if we got to a place where like from a legal standpoint, etc., I could see for rights, to your point, like this actually feels much more important, although again, it's going to be such a difficult thing to, like, make an empirical decision about. But, but I think for, like, for, for those, I can see really strongly why we, you know, intuitively would care a lot about self-awareness and intention when thinking about things like the rights of AI systems or robots or androids or things like that in the future. But for like, is it intelligence? It feels self-serving to whoever is on the side of the debate that would benefit from this to be able to say, well, it's not because like it does these things differently than a person does, even though it does, you know, imagine we have a model that is able to replicate humans in a huge variety of ways that I would like happily hire to be my intern and like I I did, you know, that replaced thousands, hundreds of thousands of millions of jobs, etc. And you're like, well, it doesn't count as AGI because it, you know, it's not— we don't have evidence that it's self-aware. Um, and I'd be like, well, but like, it's having all of the implications on society that I would have expected, um, from hitting AGI. So like, why does it matter how it works? It just matters that it works.

Yeah, absolutely. No, I, I think I, I, I'm very sympathetic to your sort of operationalist sort of approach to this, you know, like, and that's completely consistent with my take on like what intelligence is, which is just like, it's a vibe. It's like, it's a vibe. It's like what you think it is, literally.

Yeah, no, for sure. And that's like, you know, the people who are kind of more on the skeptical side saying like, we are being fooled by these models based on their fancy outputs, and that they're able to do these sorts of things. And I think to your point, like, well, if it's a vibe, then like, they're giving the vibe really well. And yes, people are like, you can probe them in different ways and show some of their limitations. And people are like, people are easy to fool, to some extent with like, with these sorts of things. So they like endow more intention and all of those sorts of things behind the actions than maybe is reasonable. But you can make the same argument that if I didn't believe I had self-awareness, like intention and self-awareness. I know I do. So I look at you, Joe, and you look similar enough to me that I assume you do too. And that's kind of all I have to go on, right? Like, I don't know any— have any other evidence that you have those sorts of things. So why— like, on some level, I get to the, like, well, why is it any different with one of these models? Like, if it acts in a way that is similar to having intention, how am I supposed to know? Well, this—

I mean, this goes back to some of the episodes that we've had on the show about defining consciousness. And so, you know, this is where integrated information theory gets into all this trouble because it claims that it, that it, it's representing consciousness when there's no way to know for sure. Or neuronal global workspace theory. So if, I mean, if we use those as a description of how— or neuronal global workspace theory, which we've talked about— if we use that as a sort of description of how intelligence works in humans, that most things are unconscious, but some things are conscious and broadcast to the whole system and sort of generally flexibly available. If we think of that as a way to talk about how awareness or consciousness works in humans, then we can say, well, no, ChatGPT doesn't have awareness in that sense, because it doesn't work that way. Yeah, exactly. We're left, we're still left with some question about are these accurate models? And can this really tell us about the stuff we care about. But we do know, I mean, the architecture is different enough so that it doesn't operate in that same way, right?

I mean, I think another example of that, of like how similar and different AI systems are to humans and how it makes it feel more and more human as you add some of these capabilities, one is like actually inspired by that book right behind you, Rolf, The Extended Mind. By Annie Murphy-Paul. We had her on the show, uh, last year or the year before. Super influential in my thinking, uh, since that time that we've had that conversation. She talks about embodied cognition, the idea that, you know, your cognitive status is not just in your head, it's actually like in the environment. Like, your thinking is happening in concert with your interaction with the environment. Through your senses and through your actions and that interplay of the environment and how it responds to your actions. Whether that environment is just your room that you're in inside, or if it's out in the woods or it's around other people. Importantly, the idea of cognition extends beyond the body, and certainly it extends beyond the brain, that's for sure. Easily, it's easy to see how cognition extends into the, into the periphery of the nervous system. And that's where, as AIs start taking on more capabilities in terms of sensors and effectors, it's going to feel more and more like they're human in that way, as their ability to, you know, to extend their cognition into the world and have effect in the world.

Yeah, I mean, a very simple example of that, that's— that comes up. So this is less in the context of— this is still embodied, I think, and it certainly fits with the ideas in Extended Mind. But if I'm writing something, there's an interplay between— if I'm writing a paper, there's an interplay between what I'm thinking in my head as I'm writing, I'm going back and looking at what I wrote, which I probably could not remember exactly what I wrote but I can see it on the page and I can, I can revise that sentence and I'll probably, you know, I might do a better job revising the sentence if I have the exact sentence from 3 paragraphs earlier in front of me and I can like look at it. And I actually think these models, like we're starting to do things in terms of features with models like this that start to look somewhat similar to that, right? Like, you know, I think one of the ways I'm excited about using I see using these models in, in tools like the ones that I work on are places where I can get a model to kind of restructure an output for me really well. So like I can pull from a few different documents and I can generate some like a table that's built off of that, that's pulling in all of that information. And like, again, it's not like the model has like memorized all that information. It has a way of retrieving it and then it's like, It's like it's like opened the page and it can see it now. And then it's going to go and take that and I'm going to ask it to restructure it in some way. And it can kind of like take all of those actions. But it really— but in order to do that, it, like humans, is getting access to stuff that's, you know, in its version of the world, which is like some set of documents and maybe the internet and maybe some other things that can look up. And so, you know, there's a similarity there. That I think is really important. And I think to your point, Joe, one might argue that, you know, things that we've seen in human intelligence like the Flynn Effect, etc., some of that has come from like we have created technologies that make it easier for people to be smarter in various ways over time. And like a lot of what is like increasing human intelligence has come out of, has come out of things like I would assume the Industrial Revolution— I don't know if anyone's looked at this. Probably someone has some historical— some historian of these things. Like, did human intelligence increase after the Industrial Revolution or as part of the Industrial Revolution? You know, the printing press changes what people can do in the world and the access to these sorts of, you know, tools over time, ubiquitous access to computers while maybe it's made our society worse in some ways and smartphones and things like like that. Um, there are— there must be ways, unless it's like, you know, enabled, um, human intelligence, uh, to increase over time, or, you know, to democratize access to certain, um, ways of working in the world over time. Um, but, uh, we can, you know, similarly, I think we're giving these models more access to more and more tools and, and capabilities that extend beyond the model itself.

Yeah, absolutely. Yeah, and just, just a callback there, the Flynn effect is referring to the observation that scores on intelligence testing since they started in the early 1900s increased steadily across the developed world until very recently.

Yeah, my understanding is it's no longer—

stabilized, if anything, maybe coming down a little bit. So at least by the measures of our own psychologist-derived intelligence tests, people were getting smarter for quite a while. Yeah, probably because of the, you know, things that they're able to learn, although there's other alternative explanations, for example, improved nutrition, etc.

Well, there's definitely other explanations.

I can imagine it going two ways too, like having access to Google makes you smarter in a way, but in another way, when you're without Google, you're a little less smart. Right.

Well, I mean, my brain has been completely broken by the internet.

So all the external, like all the external stuff you use, like it kind of makes you smarter while you're using it. But if you're tested without it.

Yeah, it's a bit— this again goes back to this like whole question around definitions of intelligence because an intelligence test has been like refined over a long period of time to measure the specific set of abilities that correlate with each other in certain ways, etc. But like what actually matters for like succeeding in the world, like looks very different over time. Right, right. Though I don't know that there's been any findings that like the performance on intelligence tests— I'm going to be careful to not go too deep into my beliefs about IQ and intelligence because I can also get quite, quite—

we get another episode. Another episode. Yeah, it's another.

Yeah, you don't, you don't want me to be on that one, but I, uh, I, I do think like that— I have no idea if the, the— we're gonna see those like correlations of those test shifts or whether they change the tests to, um, continue to correlate better over time. Because I, I do think like what it takes to succeed is changing over time. Um, and I don't know if that's in ways that are in concert with the way intelligence has historically been measured or not.

Yeah, well, I think that, you know, uh, it's a good place to sort of start Wrapping it up here, we have a last question that I like to ask people, which is, you know, what are you, what is coming up that you're really excited about? Like maybe we can keep it in the context of this conversation of AI. Like what do you, what do you see coming down the pike with AI that you're like, you just think is exciting that might be happening soon?

Yeah. Um, I think so a couple things I think are going on. So, um, I'm really curious, uh, this could be an exciting/disappointed one. I gotta be careful what I say, but. I'm, I am really curious to see how much more we can get out of these, these types of models that are basically the state of the art right now. I do think there is a take that has some validity right now, though I could be very wrong about this. So get it on tape that, that we are actually hitting some limits in terms of just— or closing in on some limits in terms of just like how much more capable the raw like transformer models are at solving these types of problems. I think there's like— I don't think we're at, you know, a complete asymptote or something like that, but I don't know that we're seeing in recent models that have come out. I'll be curious to see if OpenAI releases one this year that is is like far more capable that like we're getting the same degree of gains we were getting generation to generation over the last few years. I do think like there's tons more. It's really early still in terms of how do we use these models combined with other capabilities to solve more complex problems. So I do think like the current models have a lot more legs to solve more types of problems. And we're in a really early period there. But, but just in terms of like the raw power of the models, I'm less convinced now that like there is, you know, that's just some like magical path to like superhuman intelligence on its own just through like improvements to those. We'll see. Could be very wrong. But, but I'm excited about, I think, is that it is really, really, really early. Like, you know, last year I think everybody was just launching products that were like, well, we can have it write stuff for you, we can have it write code for you, we can do auto— like autocomplete type use cases with a little bit more to them than I was mentioning earlier. RAG has come since then, has been a big thing people are working on. And so now you have this ability to kind of like constrain, like more use this knowledge set, like retrieve knowledge from here. Um, I think like taking more complex actions and more kind of recursive, um, iteration to like automate way more things for you— I think we're in a really early stage there. Um, and I wouldn't be surprised if like we start to see a lot more of those types of things this year. That seems to be what a lot of people are, are working on in addition to leveraging retrieval and things like that. Um, but I'm just like really interested The thing I'm really excited about as we go on that journey with, with technology— again, I work in software, so I'm thinking about like, how is software going to change over time— is I think the main user interfaces and ways that we interact with, with software is going to change a lot over the next 5 years. And I think we're in a really early part of it. And like on both sides, both the people building the products are still figuring out like what clicks. And like what are the really like compelling use cases, some of which I'm very excited about. And then humans who are using software have to adapt to like interacting with software in different ways. And so I'm just really excited in this moment to be part of that journey that is like, it's going to look a lot different. It's not just going to be a chatbot for everything. Like I think we all know that's like not the right solution to every problem. I like being able to click on things sometimes, etc. But on the other hand, there are really cool things that we're only just starting to do. I can leverage text as a way of— I can generate text based on relevant information to very quickly catch you up after you get back from vacation on what were the things I cared about before I went on vacation. Or, um, there's just these, this, that part of it, just like how people work with technology changing. That's what I'm really excited about. And I think this year, a lot of it probably is going to be people starting to figure out kind of better ways to have either like tasks get automated for you that are, that are tedious or, um, uh, or, or like better ways of like you being able to kind of interact with the technology a bit less. And kind of get what you need much faster. So I'm excited for that this year.

Excellent. Well, Daniel, thank you once again for being on the show. Really appreciate it.

Cool. Yeah, it's been great to— it's been great to come back. I hope to— hope to come back for a third time someday.