Episode 44: Daniel Sternberg: The State of Artificial Intelligence

Hello and welcome to Cognition. I'm your host, Rolf Nelson.

And I'm Joe Hardy. On this episode of Cognition, we have a very special guest, Dr. Daniel Sternberg. Hi, Daniel.

Hi, nice to be here.

Yeah, welcome to the show. So Daniel and I and Rolf as well have all worked together in the past We worked together a little bit at— when we were at Lumosity some years ago, and Rolf was involved in that work as well as a collaborator. So we've known each other for some years. Dr. Sternberg received his PhD in cognitive psychology from Stanford, where he studied human learning processes by combining behavioral experiments with computational models of learning and decision-making. Uh, he's currently leading up data at Notion, and we wanted to talk to him today a bit about, uh, artificial intelligence and where we see artificial intelligence going, and particularly around the idea of what is the future of artificial general intelligence. Uh, when do we think that artificial and general intelligence may come about? As a thing. And there's probably a lot of questions that kind of fall from that and interesting aspects. And what I hope that we can do in today's conversation is explore this from the perspective that, you know, all three of us are cognitive psychologists by training, and Daniel Sternberg in particular has been working a lot in the data field over the years and has pretty good knowledge now of where we are with things like large language models and other forms of generative AI. So hoping that he can contribute there. I know also from many of our previous conversations, he said he has an interest in the philosophy of this, which is kind of the intersection that we're often working in. So Daniel, thank you for coming on the show.

Yeah, thanks for having me. Yeah, I'm definitely interested in the philosophy side of things. I'm opinionated too, so we'll see how that goes. But But I think—

Yeah, we're happy to hear any of your opinions about all this stuff too. We like opinions.

Yeah, great. Yeah, those will definitely come out along the way. It's an interesting space and it's moving really quickly too. And so my opinions are also gonna, over time, I think shift this year based on the new advancements that are coming out. So it's gonna be really interesting. I hope that hopefully this will be a really interesting conversation.

Yeah, absolutely. Yeah, so I mean, kind of what I thought might be a good, organizing question for the conversation is, when do we think that we will have an artificial general intelligence? Like, when will AI achieve artificial general intelligence? And I guess to answer that question, you know, we need to think about what is artificial general intelligence and what is perhaps general intelligence, and maybe what is intelligence. All these questions probably need to be answered. But before we get into that part, I think what would be interesting is to kind of talk a little bit about why are we talking about this today? You know, today is an interesting moment in the history of artificial intelligence where we've made some really sort of surprising advances recently. I'm thinking particularly about things like ChatGPT,, but also DALL·E 2 and other things, the whole OpenAI thing, if you, you know, and other, other folks as well. But OpenAI sort of from a demonstration perspective is especially exciting and surprising. So I thought maybe, Daniel, if you could start a little bit helping us out understand like what is like ChatGPT, what are large language models and how do they work and why is that interesting now?

Yeah, for sure. So what's interesting about— one of the things that I find most interesting, at least, about ChatGPT, about large language models in general, is to maybe go back a little bit in time first and talk a little bit about the history of these models, really, over the course of many decades. So some people might say, or the way I like to think about it is, we are in the third wave of neural networks. And by third wave, I mean there have actually been kind of three periods of time in history, actually going back technically to the '40s, 1940s, around neural networks. So the first wave of neural networks came in the '40s into late '50s. Frank Rosenblatt invented something called the perceptron in the late '50s. There were actually in the '40s some folks, McCulloch and Pitts, who built out— who were trying to build an electrical version of something that looks kind of like a neural network. This was a wave that fizzled out in the early '70s after some of the limitations of those models were found. Without going into a ton of detail there, there were certain types of functions that they could not calculate, and so they were more or less abandoned for a period of time. Second wave of neural networks was started in the very late '70s and was heavily, really in the '80s, Throughout the '80s into the early '90s with the discovery of the backpropagation algorithm, which is actually a component of many neural networks that we're even still using today. That wave was much more successful. There was a ton of research spawned out of it. Much of my exposure to neural networks when I was in college as an undergraduate and in graduate school, related to those, those models, which were able to, in theory, approximate any function, any mathematical function, but in practice were very, very hard to train to do the types of complex tasks that we even started seeing neural networks doing a decade ago because of limitations, both algorithmic limitations and as importantly, or perhaps more importantly, computational limitations. Limitations of computational power. Just as these were getting popular, these were actually the current wave of the first, the first examples of the current wave of neural networks and deep learning models were coming on just as I was finishing grad school. My last few years in grad school is where they were beginning to become popular and beginning to be shown to outperform other machine learning models at specific tasks. So I remember in particular, I remember in particular a model developed by Geoff Hinton, who was a professor at University of Toronto, has been at Google Brain for some time, that beat the best model at handwritten digit recognition, basically on a dataset of, I think it was basically from the USPS zip code recognition. It was a problem that they needed to solve. It beat the best models in class. I remember seeing him give a talk and he said something like, "These techniques that we've used to train these networks layer by layer have improved this or made it 10 times faster to train these models." It was something like this, I'm paraphrasing. But computers in that span of time also got 10,000 times faster. That made a really huge difference. And from there, we've seen this really steady— so since the late aughts, like, you know, 2006 or so, when a number of simultaneous papers came out over the last, you know, almost, you know, more than 15 years now, we've seen this rapid improvement lockstep between— in lockstep between the computational power improvements and the types of neural network architectures that can best take advantage of that computational power. And what really unlocked the current wave of LLMs, the most recent one over the last, I would say, 5 years, was the development of some of a type of model called the transformer. And for, for the kind of first decade of deep learning, when it came to language models, Generally, these models were based on something called recurrent neural networks. Neural networks that try to— In all these cases, what you're trying to do is essentially ChatGPT, the model that GPT-3.5 model that ChatGPT, the product is based on. They're all based on this idea of you're trying to predict the next token. The token is like a word, but it's not exactly a word. It's basically an exhaustive set of combinations of commonly occurring letter strings. They're all trying to do that, but for the first decade or so, these models were recurrent neural networks where they were trying to actually predict a sentence token by token. You can think of it as you have one sentence, you're predicting the next token each time, and then at the end you go back and you change the weights in this neural network to, uh, based on all of the, those inputs. What transformer models do that is different, the kind of intuitive version of them, is they actually do this all in parallel, um, and they are using, uh, something called— they're using attention. So basically attention gating to focus on the next element in the sequence But they can do it all at once. And the reason that is really powerful is that, without going into a ton of detail about it, is that it's way more efficient to train. You can train it using GPUs, which is the— which is where we get a lot of the computational power from these models. You can train them way faster because you're training large amounts at once. And so they enabled models like GPT-3 to be built. Which could never have been trained using LSTMs or other types of recurrent neural networks that were popular before that because it would have been so time-consuming and computationally expensive to do it. And so what this has meant is they've been able to radically scale up the size of the models and the amount of training data that they can be trained on over the past, over the past 5 years. And so, you know, GPT-3. So first off, stepping back, ChatGPT, is technically based on a model called GPT-3.5, which is basically GPT-3, which is a model that came out, I want to say, 2 or 3 years ago, um, with some additional tuning for specific types of tasks. Um, and GPT-3, just to give a little bit of background, uh, has something like, you know, this may not mean much to people, 195 billion parameters. Which is basically weights in the model. That's a lot of fucking weights. So, it's a lot of weights. I was reading somewhere they need approximately 700 gigabytes of memory just to store all of those weights. And it was trained on— oh, I had this somewhere. It was trained on many terabytes of data, let's put it that way.

So you can't just run this model on your computer at all.

You cannot just run this model on your computer. You can definitely, you know, if you could— it's not open access, like it's not open source. You have to— OpenAI developed this model. They own the model. They, you know, they are running the model on, I assume, very large GPU-based clusters with a lot of memory. There's a lot of interest, I know, in this field in like figuring out how to smartly prune the models so you can run like you can take a trained model and run a less memory-hogging version of said model, but that is— that mostly maintains performance. But I know that's a, you know, a field of active research.

I wonder if— well, I don't— I don't want— we can skip back. We can go back to some of this stuff. But I wonder if we can start thinking a little bit about how— I mean, these models were inspired by— originally inspired by neural networks. I mean, it was Hebbian networks, and Rosenblatt took that kind of stuff up and thought about how to make a digital version of it. And some people care about the similarities between neurons and neural networks, and some people don't. If they work, they work, right? But I wonder if you're, you know, from your perspective, you have a good understanding of, you know, both cognitive architecture and neural architecture and and network, these neural network architectures. So what do you see as, I mean, you said pruning, you know, that sounds, that's a neural, you know, sounds like a neural term, right? Neural pruning that you're getting rid of some connections. You talked a little bit about some attentional gating that might go on, which sounds like, you know, that sounds like cognitive function. What do you see emerging, I guess, in neural networks that looks similar to human cognition or human brains?

For sure. I mean, and, and I come at this from a slightly biased perspective based on my training, and I want to say the labs even that I was in as a graduate student. Um, I think, I think the most—

the—

at the most basic level, the similarity that I see between the way these models learn and the way humans learn at the most basic level is that they are learning about the patterns and statistics of the environment in which they live. I don't want to use the word live.

We live in a 3-dimensional world and they live in a world that just receives data from some input, right?

Exactly. We have sensors that they don't have. We have, yeah, we also have—

Effectors that they don't have.

Yes, exactly. Effectors that they don't have. So they're in this very impoverished environment, right? So if you're a GPT-3, you are just getting tokens of input given to you. But you're trying to learn the statistics of those. And obviously the person building the model is thinking about the problem they're trying to solve, and they are constructing, um, the infrastructure of the model with components that they think are well suited to that task, to that statistical learning task. And so I think there are some basic algorithmic aspects of how those models work, some of which are pretty reasonable potentially approximations of neural processing, sort of at a very basic level, in a way much simpler though. So for example, the simple idea of, you know, you're doing some calculations between input, multiplying that by some, you know, it's matrix multiplication basically with a bunch of functions, nonlinear functions attached to it, attached to the outputs of those.

Which is a good enough approximation of what a, neuron is signaling.

Sure. Yeah, that's a decent approximation of what a neuron is signaling or some set of neurons is signaling. They may have, you know, I don't know that they have the exact same characteristics of neurons. In fact, I'm sure they don't. But there are other parts of it that are like arguably less neurally plausible. So for many years, people have said things like backpropagation, which is critical to learning in many of these models, is not a biologically plausible idea.

Sure, the idea that we have some direct contact with ground truth in the way that we're— in the way that, you know, these data structures have a definite—

Exactly.

And when you feed— and when you train like a very, very deep model, and when I say deep, it means there's many, many layers of matrices multiplying on top of each other, essentially. And you need to train, you need to recalculate what those should all be based on the ground truth information, and then propagate the error back and tune the weights of these models through many, many, many, many layers. It's not like when I, you know, I do something in the world and then my eye gets output, then it like gets to send error signals all the way back through the system again, through many, many layers of neurons. Right.

Obviously respond with much less feedback often, you know, a few trials and things like that. Yeah.

Although the models are getting better at that. Yeah. Quite a bit.

That's an interesting thing to see.

But yeah, so I think there's the— when I was in grad school, so my advisor was— my advisor, whose name is Jay McClelland, was a pioneer of neural networks in the '80s, and specifically around neural networks for modeling human cognition. So he was a psychologist by training. And worked with other psychologists and also with computer scientists.

Wrote the famous Parallel Distributed Processing books too.

Exactly, exactly. And it was, you know, he was very, he has been, or at least was at the time, very interested in, you know, the analogy of these models to human cognition and how we learn as a way of more like pushing the boundaries and understanding how you can— what types of cognitive behaviors that we take for granted in humans can be developed using statistical learning. And so I always thought of it mostly at that level, that he's not saying this is like, you know, we might be interested in more neurally plausible models as well directly, but more of this idea of, you know, after the kind of cognitive revolution of the '60s and '70s, You know, what is it that we can learn statistically without having to make assumptions about rule learning rules, innateness of specific concepts or capabilities? To me, it was really all— at least my interpretation was often this was always this is about how do we, how do we understand what we like? Surprise ourselves in a lot of ways with what we can learn through simply measuring the statistics of some input. You can get surprisingly far doing that, which is one of the reasons that I've been really— One of the things I found really funny in the discourse right now around these models is there's a very like the most naive criticism I've seen of consciousness, sentience, whatever in these, whatever you want to call it, from these models is, yeah, but they're just learning the statistics of their input. And when I hear that, I'm like, you have no idea how we are learning or processing information in our brains necessarily. So why are you— why would we assume that learning the statistics of the input is not how we also learn many things? The input, I'm sure, looks different.

Yeah, I mean, in some ways that's, that's partially learned evolutionarily, right? So some of that quote-unquote learning is something that's built into our system, right? Like, yeah, you've got effectors and sensors that have been co-evolved with our environment to, you know, take in the, the important aspects of the— that's part of the statistics, uh, statistical space that's relevant to our survival, right?

For sure.

Yeah.

And maybe just to draw out that analogy a little bit further, I agree with you, and the parts of that are the things that are stable in our environment are things that we can, if it makes sense to, over a long, long period of time, actually encode evolutionarily, like as a bias in our learning systems in some way, right? So like some people, I would say, might think, yes, you can learn the statistics of natural scenes, but natural scenes look a certain way no matter what, because that's just the way the environment of Earth looks. And so we might, over very, very long periods of time, encode biases toward that, even in our, even in, you know, early visual cortex or something like that.

Well, that's a place where you could see that machine learning— I mean, if this stuff's only been around a couple, you know, couple decades, it's hard to fault them for not catching up to the amount of work that evolution's already done, right?

I mean, 100%. And the other thing that's funny, right, in that analogy is that if you look at the models, then there's like two— there's two things going on. One, they can like potentially learn things that we would learn evolutionarily, but they have to learn it through training. And then two, Unlike— well, I don't want to get theological here, but unlike us, potentially, there is a creator who is trying to, as I mentioned earlier, design these models to have a structure that makes sense for the problem at hand, whereas we need to evolutionarily learn— get to that state, right?

Which is more or less blind design, right?

Yeah, exactly.

Yeah, well, I think that's— yeah, there's a lot there. I mean, I think that, you know, that kind of brings it back to the sort of organizing question for this conversation. You know, it makes me want to, you know, dive in a little bit more on what are these models good at today and what are they not good at and what are humans still good at that that machines are not good at and what are machines actually a lot better at than humans are. And maybe it makes sense to talk a little bit about what we think about as what we mean by intelligence or general intelligence for humans. And then maybe that can inform a little bit what we mean by artificial general intelligence. You know, in general, when it comes to intelligence, We think of— I mean, I've made this argument on this show a few times, but generally speaking, when we say intelligence relative to another species or relative to a machine, we basically mean, does that thing think like us? Right? Like we grant intelligence to a thing the more it thinks like us. I mean, that's in a very basic way, and that's maybe not ultimately the best model for thinking about what intelligence is, but that I think from a colloquial perspective, that is what we mean by intelligence, is things that think like us, because forever and ever, we've always been the smartest things around. This starts to break down a little bit when we talk about machines that may be more intelligent more intelligent than us or super intelligent. They're already super intelligent in certain domains. But in general, some of the characteristics of intelligence, when I talk about, you know, here, human intelligence, which is a capacity for learning, reasoning, understanding, and aptitudes for grasping truths, relationships, facts, and meanings. This idea that, like, it's inherently tied up with the idea of flexibility. Especially general intelligence, the idea that you can be flexibly solving problems that you encounter in your environment and learning from relatively few examples of how to solve those problems efficiently and effectively. That's sort of how I would define intelligence for the sake of this conversation. I don't know if you guys have inputs that you want to throw in there.

Well, I think the comment on intelligence is interesting, right? Because as you mentioned, machines have been— there are many things they've been better at than us.

Right?

So, you know, any computer can do all kinds of mental calculations— well, sorry, mental— but calculations that I need to try— that I could never do mentally myself.

Yeah. For example, you know, math, you know, like basic, like addition, subtraction, multiplication, all those, right? Super, super fast. Way better than us. Have been forever. I mean, since the calculator, right?

Exactly. And I think, so I think when we, if we say, is a machine intelligent at X, we may mean, yeah, are they very good at that task, or do they do that task in a way that is similar to us or better than us?

Yeah.

I mean, so one frame for talking about, so I guess then general becomes the this is sort of one of the important phrases here, right, or one of the foreign words in this conversation, because the idea of general intelligence is that if you're good at one thing, you're actually good at other things as well. So what— in psychology, general intelligence refers to this factor. So there are all these models that basically analyze cognitive abilities and variations in cognitive abilities in populations across different people. So What they find is that, you know, there are different factors so that some people are good at, you know, people who are, you know, you can break down abilities into different subcategories, if you will. You know, verbal ability, spatial reasoning ability, these sorts of things pop out as factors statistically in the analysis of people's cognitive abilities and performance. And general intelligence pops out as sort of like the primary principal factor that actually explains a lot of the variability across all these different capabilities, like spatial reasoning and verbal reasoning. And it turns out that people who are good at one actually tend to be good at the other. So there's this characteristic of what's called G, or general intelligence, which is that people who are good at one thing tend to be good at other things cognitively. And people who are bad tend to be bad. I mean, it's a statistical thing, so obviously there's tons of exceptions and everyone's different, everyone's unique, but there's this overwhelming factor which is no one has really sufficiently explained to my satisfaction. And I don't know what the answer is, why that works that way. I have some theories, but, you know, there's this G factor that sort of dominates all the other factors so that if you're good at one thing, you're good at other things. That's general intelligence. Now, general intelligence in the machine learning situation just means really just transfer, that you can get good at one kind of task, but then you can actually solve another type of task as well. If your whole world is just solving one type of simple task, then you don't have anything like general intelligence. You've just got a very specific intelligence. It may still be super intelligent, But in a sense, it solves that problem better than humans. But you're not generally super intelligent.

I think one of the things that's particularly interesting about this moment with the demonstration of something like ChatGPT, which again is based on mostly a model that's existed for a few years and they all say GPT-4 is coming. I can't say more than that, but it is that it takes a problem that sounds very constrained, right? The model is trained to predict, like, to predict tokens of text, right? Go from here, take a token as input, predict the next token. And then what it's trying to do is it's, you know, it's processing through everything you send it when you send it something to ChatGPT, and then it's generating for some amount of time. And that, that problem sounds very constrained. It's just about predicting the next token. Yet, you know, as of I think the last, the last news articles I read related to this were telling me GPT has— ChatGPT has passed a bar exam. ChatGPT has has gotten a really high GMAT score. Um, there's like a bunch of, uh, examples of like a high enough score to get into the top business schools. Um, there's a ton of examples of this right now of on a variety of types of tasks that it's able to do well at.

And, um, this is why it's surprising now, right? Exactly. Surprising now. It's not just answering some very basic question in one very specialized domain. It's like anything that you can type, you know, there's a decent chance it's going to come back with something pretty smart, frankly. I mean, it's, you know, just this past couple weeks I've been using it in different ways just to explore where it could be useful already. And I was doing some programming problem using a Squarespace website, you know, some very specific, you know, not interesting domain of knowledge, but like, you know, asked it like to solve this problem, you know, for how I code this JavaScript thing for a Squarespace website. It had the answer and it produced it in JavaScript, perfectly formatted, ready to cut and paste and just throw into the project. You know, asked it to, you know, create a dialogue in Spanish for, you know, about a trip to like Titicaca that a father and a son might have because we're planning this trip to Bolivia and blah, blah, blah. So I want to have this conversation with my son. We want to practice Spanish and it's like, oh, I can then translate it back to English. So it creates this whole Spanish lesson essentially and does that perfectly in like 2 seconds. These are very different kinds of problems and it's able to solve them in interesting ways. Um, it's surprising to me that it happened this year. I wasn't ready for it. I mean, I was like, it is, it's, it's, it's, it's general in ways that I find surprising, I guess.

Definitely. And I think one of the things that's interesting too is, you know, I want to maybe briefly just to break down the components a little bit. I think this is relevant for the, the topic of thinking about intelligence and how intelligent the current generation can be. Even like we could talk in a moment, I'm sure we'll talk about what this looks like over the longer term as we, as, as these, the technology itself gets better. But even if you just take one of these models, you can do a lot of things to give it more power. So it's worth thinking of ChatGPT as a user interface with particular components on top of this language model, right? So I have a language model. OpenAI is presumably— when you send something to ChatGPT, they are actually seeding it first with some prompt. So there's a whole topic right now around these models. People talk about prompt engineering, which is basically generating, like, learning how to give it the right text to do a good job at different types of tasks. And they are setting it up to do a particular type of chatbot task. Some people even tried to reverse engineer by getting it to give you back what the prompt it was given was. Um, and, and so that's a particular interface. You can give these models more and they can get better. So for example, imagine, um, I have some large base of knowledge that I want to give, uh, that I want ChatGPT— I want GPT to be able to work with. If I have some way of, you know, for example A really crazy example is imagine I— it essentially can tell me what it wants to search for in Google. I can give it information back, and then it can use that to, to help improve its answers to questions. I know a lot of people are saying, you know, what's built into ChatGPT is in some ways better than Google, but still, imagine you can give it access to specific information. So more recent information. Um, you can potentially get these models even in the current generation, with a little bit more fine-tuning potentially, or maybe even some cases you don't need to train the model further, uh, you can— if you add additional bits of, you know, like applications on top of it, you can make the model much more powerful. So I could ask it to, um, you know, go get me, uh, the most recent news articles related to some topic. Because if you've ever used ChatGPT, it actually doesn't have much information since late 2021 when that model was trained. But I can actually have it search for that information and I can provide it back to it. Now if I have a search interface, I can train the model to use that search interface. I could train the model to respond in a particular format that I can then put into a computer and have it do something and give it back output. And so I actually think these models are, you know, if we're surprised right now, I think this year a lot of what you're going to see that's going to continue to surprise you is less, oh my gosh, there's a more amazing language model. That will probably happen at least once this year. But in addition is all of the things that people put on top of it that kind of enrich its ability to do things with its current understanding of the statistics of language and knowledge and all of that. And I think that's going to be even more surprising for people. And again, it's about enriching the environment that the model has access to, right? There are a lot of things I can do to make myself seem a lot more intelligent because I have the ability to, you know, go get new information for myself, for example.

Yeah, it would be— I mean, it's also interesting to think at this point how these— how the architecture of these systems is different than than our cognitive architecture. You know, so what process is it going through? And, you know, we understand something about how, you know, human reasoning and judgment and all of that works. And it's, I mean, it's very different from the way that information gets processed through a neural network. You know, for one, there's not a continuity of processing, you know, as, you know, we would have a stream of consciousness that that continues on over time and that we're aware of, whereas there might be a lot of discontinuities between, you know, processes that are going on in a computer. And, you know, other sorts of differences that might show up in cognition too, references to real-world events or reference, you know, an answer to a question might come from an autobiographical memory, something that you've experienced before versus just a database that you're grabbing from Google. So there may be, you know, a representation of yourself that may not exist in the system too. So, you know, any other sorts of— how do you think about, you know, you're thinking about how's the brain thinking and how is this other system thinking? How do you conceptualize that?

I mean, there are some things that these models have And that they've been built over, you know, as people have, as researchers have iterated on them over the years to improve them. That in the structure of the current generation of models, my understanding at least is, and I should caveat here, I have not, you know, obviously I do not have the many millions of dollars to go and build one of these models myself from scratch. It is extremely expensive.

You need a lot of computational power.

Yes, correct. That's why you're seeing specific companies standing up that are funded just to build these and maintain platforms for them. But what I was going to say is there are things that have been— there are components that have been built into these models that actually, I'm not going to say they remind me exactly of how our brain works, but the attentional gating, the need for in the prior generation of recurrent networks, some like, uh, ways of maintaining and persisting something in memory. Um, and then also actually they— though that generation had a, an active forgetting process as well that were needed, was needed. Um, those are, those are things that, you know, are purported to exist in brains or something similar to them. Um, and so you do end up creating some things, some structures that you actually need need to do these processes that are at least in some level analogous to the way a brain works. But I agree with you.

And you could see like attention, of course, is, you know, for a brain, it's a way to be able to process the best information and ignore the worst information. And for a neural network, of course, it's a process to save computational Power, right? That you exactly focus on what you need.

But, but I agree with you in that, like, you know, you don't see the same kind of grouping, at least that I've— in my understanding of the models. And I would have to think about if there's some analogy there I'm missing. Um, something close to like, well, we have this slow learning process through— in fact, in the brain there's many, there's many of them, but, um, you know, there's like, uh, like long-term memory consolidation There's a reward-driven learning process. There's consolidation and faster episodic learning mediated by the hippocampus. There's these different learning systems. They're interrelated, but they also work differently. This is the thing I was interested in actually when I was in grad school and related to the research I was doing, was trying to actually learn about how we could tease apart those learning systems even just in behavioral experiments, potentially based on the demands of the task, which was a pretty fun— well, really nerdy but fun exercise back when I was in grad school. And I think, you know, again, these are— these models are much more tuned around or designed around the task that they are designed to— the problem they're designed to solve. So autobiographical memory, arguably for the purposes of what they're trying to do with a, you know, large language model today, is not a critical component.

Yeah, it's not going to be alive long enough for it to really matter. It's because the next generation is going to come along.

Yeah, I think one of the interesting things though to me in that, like we mentioned how we've been talking a lot about how we're so surprised by the way, the surprise comes from this like model was just trained to do this training on, to learn to predict the statistics of language. —on the language input it got. And it does all of these things that are very surprising that look like general knowledge. And I think one of the more intriguing aspects of that to me that makes me think about the psychology—go back to the psychology world—is like, what is it that I assumed mechanistically had to be built into a system versus like is actually in some way just afforded by learning over time in some—

Statistics of the environment.

Yeah, the statistics of the environment paired with the right kind of inputs for a particular task.

Yeah, I think, yeah, we should, we should like focus on that, I think, a bit because it's, it's, it's kind of really interesting to think about. I think, you know, just what are the inputs that this thing has? What are the outputs that it has? Like, what are those, you know, sensors and effectors and how does that relate to what it can do. And I agree, I think that's why I think most people were surprised. I think it's just that, not that the model is performing so much better than it was before, but just that you can engineer a system that is so useful and interesting to use with the existing systems, doing it with the right engineering. Like, and that's about— it's about the user interface. To your point, it's like creating the environment that the user can communicate with the machine effectively in something like real time. I mean, I think that's the real innovation that OpenAI has, has, um, yeah, yeah, exactly, revealed in the past few months.

Yeah, I don't know if you've heard recently, you know, very recently over the last few weeks, there's been, you know, there's a lot of— I, and I don't want to, you know, I, I know people, these companies, but like there's a lot of like seemingly sour grapes coming from Meta and Google and some of the big bear, you know, big tech companies around what's out now because they, they're like, look, we've had this technology internally for some time. We've had versions that were just about as good as this. But what they haven't done, one, like with ChatGPT, they package it into this like product that they've been willing to launch. And I think OpenAI is willing to take more risks than some of the other folks are. But in addition, they, they haven't given people the building blocks to work with these models themselves. And I think what you're starting to see right now is that with— and OpenAI is not the only company, there's a number of these companies out there now that are fast on their heels, but is that they're kind of— they're creating these like platform and developer components to like make it pretty impressive, pretty easy to build pretty impressive things into applications of all kinds. And it's very different than, you know, Google giving you access to some like cloud compute thing where you can deploy a model. This is literally like just, you know, send us text and we'll send you something back. And so you can build some pretty, pretty amazing things. And yeah, I think I agree. I think that in terms of what they have access to, again, it's this very— in theory, it's this very, you know, impoverished All I get is tokens and all I give you is tokens.

But, but it's also like, where are those tokens coming from, right?

The tokens that they're being trained on? Yeah. Yeah. It's a huge volume of information that's been scraped from the internet in various ways.

It's the internet. It's the entire internet, basically, right? Yeah. I mean, just about, right? Like, if you think of it, it's like if it was— if you could Google it like then ChatGPT essentially might very likely have access to that information if it was before 2021, right? Yeah.

So apparently it was trained on 45 terabytes of information from the open internet. Um, for many— so, and I mean, I'm looking this up right now, it says the, the datasets, and they don't describe exactly what they are. Common Crawl, which is 8 years of web crawling. Uh, WebText-2 is the text of web pages from all out outbound Reddit links from posts with 3+ upvotes. So not surprisingly, a lot of Reddit. You might get that in some of the ways that it talks. Sometimes, although you can tell it to talk differently. Books One and Books Two, which are some internet-based corpora of just like complete text of books. And Wikipedia, like all of Wikipedia. And so it's, yeah, it's like basically all of the information on the internet. And it's also worth noting, the other thing that's worth noting here is like, it is 45 terabytes. So it's true that it can't actually store all of this information in the like 700 gigabytes of memory that it essentially stores in the parameters of the model. But it's much closer to that than for sure any image model is in the sense that like it really does seemingly memorize Bits of text for sure. And so like, because it is the parameters of those models are so huge. And so it's the reason I say that images can't work that way is images are just much larger. Text is easily compressible and images are— can be compressed obviously, but the amount of information in an image is actually much higher. And so like image models definitely cannot do that. It's an interesting topic around copyright, which we don't need to go into today, but there's a whole, a whole world around that right now. But yeah, I think, you know, it is the text of the internet. It's the internet that they're trained on. So imagine a person who was just fed the internet for their entire upbringing.

What a miserable person that would be. Exactly. Yeah.

Who couldn't do— and also, you know, at that point now they may get trained GPT-3.5, which is what ChatGPT is running, is basically that what we just described, plus it's trained on—

it is—

they actually had tasks that they had it do. I don't know the details of those tasks. And, and they also fine-tuned the model quite heavily based on, based on the desired output for those tasks. So for example, you might, you know, I think there are versions of GPT-3 Plus whatever that are, you know, trained on more code, for example, or writing code. And so they actually try to get it to be better at doing certain types of instruction-based tasks. But the fundamental model is the same. They're not changing the model. They're just fine-tuning the weights of the model.

So I'm going to ask a sort of naive question, which I've always sort of wondered. Which is, so, I mean, you can have something that's specialized in this area and something that's specialized in that area. Why is a general intelligence not just sort of a Swiss Army knife of, you know, all of these different skills plus some sort of way to move between them or decide between them in the same way that our brains are somewhat modular and contain bags of tricks that are sort of interconnected in ways that we can, you know, so for facial recognition, we've got a nice area that can be specialized at that.

Well, what's—

I feel like there's something wrong with the way I'm imagining this, but what's wrong with the idea of a sort of an executive function system that farms out a lot of duties to other things when they, you know, Google something when, when it needs to, and sort of uses other things as tools in the same way that we use specialized processing in our brain as tools.

So are you saying, like, I mean, I—

the bag of tricks kind of thing?

Yeah. All right. Yeah. Okay. So you're imagining, like, if— what if you just had different models that were stitched together by some other model Yeah. All right. That's got to be where it's going, right? A little bit.

Well, some things are kind of like that already, right? So DALL·E is essentially a combination of models that were then trained together. I— and I'm not an expert in the structure of those models, but essentially you are taking a language model and an image model. You're putting them together. The language— like, what's— which one of the— right. One of the, like, most critical parts of the models we have, these, these newer, this newest generation of models. There's a concept that, you know, we now talk about as fine-tuning a lot. A few years ago, the term I remember people using was called transfer learning. And the idea was like, I take a large language model, all it was ever done to train to do was like, you know, predict tokens. Now I want to use it in a particular, very specific context. What I could do is I can take that model, I I've already got all of the weights trained based on this huge dataset. I can't possibly train it. I don't have the resources to do it. But now I have the specific task I want it to be really good at. I can add a layer downstream of that too, and I can just train that layer without backpropagating anything to improve at the task that I want to do. When you take something like DALL-E, you essentially have, well, there's a large language model that was trained on language, so it just understands language. Sorry, I don't want to get anyone mad at me for saying understands language, but I mean, we know what you mean.

Bear with me for the moment.

It's, it's very Marcus Universe language. Yeah. As long as Gary Marcus isn't listening, I won't get in trouble. And, and then we have a model that's trained on images and then we have a dataset that is, you know, labeled image, labeled images. And that allows me to essentially learn this relationship between these. But I don't have to learn language over again. I don't have to learn the structure of images over again.

That's— yeah, that seems to be so much of the, of the beauty of having this in digital format is that you can just copy and, and use that, use that same snippet in, right, and insert it into a bigger system.

Well, this starts to get it. Sorry.

Oh no, go ahead, go ahead.

I was just going to say, like, this starts to get into this question. So like, you know, we start off asking like, when will we have artificial general intelligence? Now if we take like the Nick Bostrom definition of superintelligence and say AGI is when computers are just better at everything, that's everything sort of relevant, that everything that matters, you know, that we care about, just computers just better at everything, getting to that stage somewhat quickly I think a lot of people have opined, and it sort of makes sense from like a, you know, science fiction perspective. Like, what you really need is like computers to make better computers and algorithms to make better algorithms and so on and so forth. And it feels like that is where this wants to go next, right? It's like you want to tell the machine to make itself better. Yeah. And I mean, I think that's, you know, that kind of to me also ties in with this question of of the environment of what this algorithm is allowed to know and allowed to experience and allowed to affect. So it's like, I think a next big leap in terms of moving towards machines that really are making themselves smarter is they need to be able to allocate resources and affect resources in the real world outside of just their clusters. That they've got access to directly.

Are we creeping towards robo-pocalypse right now? Yeah, definitely.

It's an interesting question. I have like, this is why I mentioned earlier, you know, I mentioned before we started, I have opinions. Let's hear them. Well, but this is an area where I have like very conflicting opinions and I'm not sure. There's a part of me that thinks of this as kind of a historian. And the historian in me is making an assumption— it assumes almost that we are going to hit some kind of large, uh, technical wall in the next few years, some at some point, because we always have. The history of AI is like— winter, right? It's like—

and then, and then, right, I— winter, and then AI winter.

And I don't think it would necessarily be winter this time, but I do think it— there's a world where maybe we just— Early spring. Yeah, we need to have some next leap in— there's some fundamental aspect of these architectures that is limiting and will not allow us to get there.

You don't see inertia as being so— or momentum, I guess, as being so strong that it's inevitable?

The singularity is approaching. Yeah, I'm not—

yeah, I also— well, I'm going to avoid talking about the singularity for a moment. We can talk about it later. Singularity. I like— so that's one side. The other side of me with this, going back to the surprise that we've kept talking about, is, as I mentioned earlier, I actually think, yes, training, you know, bigger models, training them, better types of training for them. I've actually heard some people say that they don't necessarily think the models just need to get bigger and bigger and bigger. There's some things that we can do more efficiently as well. If you give them better input that is more tuned to specific types of tasks, you can actually train them much faster and train smaller models potentially. But I think the more important thing right now that's going to, again, I think lead to more progress sooner is we have really only scratched the surface of what we can do with the models that already exist. If we actually give them a little bit of fine-tuning and access to effectors that they don't have access to right now. And when I say effector in that case, I don't mean a robot operating in the world, because like, you know, if you want to talk about things machines are bad at, you know, walking— walking, yeah, walking is a really good one. Picking up a cup off a table, right? Um, those are actually still really hard. And the reason they're still really hard is actually because The— here, the, you know, the nice thing about language models is that most problems can be re-instantiated as language, and language is, is like, is low information from a, you know, from a computer standpoint per unit time relative to what I need to do to move an object in the physical— in, in a physical—

to get feedback. And yeah.

And get all the feedback. Yeah, exactly. Um, so that's— I think, but, but in the context of a computer, um, or a computer that— think of it as a computer that lives on the internet, uh, there's a lot of things I can give it access to do, and there's a lot of things I can do where it can get feedback. I actually think it starts to be useful to think of this problem as more of a reinforcement learning problem. Um, so reinforcement learning problem where you You have, there's a state of the world, the model's observing the state of the world, it's taking an action, it's observing how the state of the world changes. While I can only give it information in the form of text, I can give it all kinds of text. I can say, you have 3 things you can do. You can look this up on, you can do a Google search, you can write this code in this language, or you can name a third thing. If it can choose one, it can choose it in text. I can then give it a task to do. It can do that task in that text and I can give it feedback on what happened. Even just adding that in, you start to see an ability for these models to do some pretty potentially impressive things if they know how to do it. ChatGPT can write code decently. Well, what if I actually give it access to an interactive engine for running the code that it wrote and getting output from it? Really obvious example. Thing that most computers are good at that these models are terrible at is basic math, which is ironic and fun in a lot of ways. If you'd ask ChatGPT to multiply two 3-digit 3-digit numbers, it's going to often give you a result that looks semi-reasonable, but when you calculate it, it's off by some amount. It's actually terrible at that. But obviously, I could give it a calculator. I could— No reason why you couldn't give it a calculator.

Just let it run that piece of code. Yeah. Or actually— A little Python window. Exactly.

That's the other funny thing. That's the other funny thing. You can ask it to write. I played around with this once. Just funny example, I, you know, ask it to calculate compound interest as a fun one, right? And it does it wrong. Like, it— and you can even tell it to walk through its steps. And that was actually—

that came up the other day in the New York Times, right? They were— there was like, um, some— oh, CNET. Yeah, this article that was written by ChatGPT actually, and it was— it got compound interest wrong of what it was. So this is—

I mean, this is a place where artificial intelligence is interesting when it's making interesting mistakes like that. Because of course, in, you know, in people, we understand a lot about cognition from the patterns of mistakes that we make. And when we, you know, it tells us how we operate in normal circumstances. I mean, so that, that's a fascinating thing to me. Is there any, anything else that you've noticed about ChatGPT, toying around with it and sort of seeing capabilities, what it, what it Yeah, I mean, that gives you a sense of how it's, how it's calculating that stuff. It's not doing it like a calculator.

Yeah. I mean, if you give it very simple numbers, it does it. But my assumption is it's almost memorized them. I could be wrong.

It's similar to how a person would do it, right?

Exactly. And what— but, but I think, you know, we're talking about getting to superintelligence or, you know, AGI. Well, in that world, if I'm just trying to say, how do I make this like superintelligent machine? I totally should just let it run Python code because if I ask it to write a function for generating compound interest in Python, it will likely write it correctly. So if I just let it run it and run things through it, it will likely do it correctly. But if I ask it to just explain it to me using its neural system, it cannot actually do it. So it's a— it's an interesting— I'm trying to think of other examples, though, of what it's like particularly bad at. I don't know, Joe, if you've seen any.

Well, I mean, you know, there's the things about like, you know, Sometimes, you know, if you ask it for recommendations on a movie that's appropriate for a child, for example, it might get that horribly wrong and you might be exposing your child to like a movie that you don't want them to watch. That happened to me the other day. Actually, I don't even remember, but I fortunately previewed it myself. Right. Just, you know, anticipating that they've also never—

it's never seen a movie. Right. So that's right. Never seen a movie.

Doesn't know about like— yeah. So there's this, but I mean, the things that it really can't do or anything like, yeah, anything that requires moving things around in the world. World, right? Like affecting anything in the actual world outside of this screen that's right in front of you, right? Like it's pretty good at putting words on the screen and not very good at other things outside the world. But that's where, you know, it's interesting, you know, I got thinking about this as we were talking, like it's really bad at like going to the store to get some milk. But what if you said, all right, you can actually have access to my credit card and the internet and you can, you know, tell Postmates to like, go get me some milk at the store. And like, suddenly you're asking it to solve that problem. And maybe then it's just simple, like, oh, okay, well, give me the best price on getting the milk from the store in this amount of time. I mean, it's going to be able to solve it. I mean, you could, you could engineer a system today that would solve that problem pretty well. Definitely. So, you know, That's where you start being like, well, we give it the credit card, tell it to make itself better, give it access, let it give instructions to humans who are willing to do it for money. And suddenly, you know, you've got a system that can build better versions of itself. Yeah.

To the extent when we say intelligence, we mean, you know, performs like us in the world, not just like can do cognitive tasks. When I like give it to it in its environment in a way that makes sense in its environment. But like, you know, an obvious example would be I mentioned the, the like passing a bar exam, but it's like, well, it passed a bar exam in the sense that like I fed it the text of the bar exam and I asked it to give me answers to questions. I didn't like, you know, give it like make it like sit at a desk and look at a piece of paper or something like that or look at a computer screen. But to the extent we mean like, oh, it has to do it with the effectors and sensors that humans have in the equivalent way, then yeah, like that, then it's nowhere near. And I don't know, that could take a very long time.

We need good avatar suits and we just pop our AIs into them and let them run around the world. And I mean, they should get a lot smarter if they're interacting with the actual world.

Well, yeah, I mean, yeah, that's one thing where, you know, One of the things that they're not good at is understanding the customs of, or like ways of interacting in a specific subgenre of the world that is not just like reflective of the broad internet. So it's like letting it go out into the world and experience those things and how to interact in those environments is one of the ways that it's going to learn. Yeah, exactly. But no, I mean, yeah, exactly. I think that so in terms of getting to artificial general intelligence, maybe it doesn't need to go through recreating— like, does it? Maybe it doesn't need to go through robotics. I guess that's, that's sort of the question. Like, does it? Because like, if you need to have the same effectors as we have, then it's going to take a long time because you have to build a robot that moves like a person, which is maybe closer than we think. But, you know, as we know, it's hard. Yeah, I mean, tell a person to do the thing and then feed that back into the computer. Yeah, maybe it gets there a lot faster.

Yeah, I'd be very curious to see, um, do we see major advancements in robotics in the next 10 years? I mean, this is an area I know very little about, like very, very little. Um, and so I assume, you know, um, robots and companies that are trying to train robots are using these types of models in some ways. On the other hand, To the extent that model, you know, there's, there's all, you know, you can think about it, certainly self-driving cars, which you might think of as being like robots, are, are definitely using deep learning models. And I don't know off the top of my head what they— how much of that is happening at the edge, which means like in the, in the, in, inside of the car itself. Hopefully in your car it's all happening inside the car and not relying on a cloud connection. But presumably, you know, a humanoid robot You can't actually have the level of computing that even can be in a car because a car is quite large and has access to like a really intense power source. A lot of battery. Yeah, a lot of battery. Whereas you can't have that on like a humanoid robot. And so there's, there's like actual fundamental hardware limitations there that, that again, and you know, that's one of the powers of these just pure language models is all they're trying to do is language and they can live on gigantic beefy machines in a data center somewhere. And so they can be pretty powerful. But then obviously that's not going to work in the real world. But I think, yeah, going back to your point, Joe, I agree. I, I think we probably need to— we are probably in the early stages of figuring out and debating what we even mean when we start to talk about intelligence, because there is one— there are different versions here, and maybe we need to open our minds a little bit to the idea that machines are going to be able to do a lot of tasks and do them in a very different way from us. And I don't mean, you know, there's very different way because they are— because of the architecture of the model, but there's also in a very different way because, again, the sensors and effectors are all just completely different than ours in the way that they're getting the get the task done. But nonetheless, they are the, um, they are the way that the computation is actually happening, right? Um, like, like, Joe, in your example, yes, you could go get a person to then go get the thing for you, but, um, but, you know, you can think of it as kind of like the Searle Chinese Room kind of thing, where, where the, the artificial intelligence is the, the computation happening in the room, and then the, the, the person going to get the thing is just performing a rote task. And that is, you know, literally just, you know, we need you to swipe the credit card or we need you to walk over there and get the milk, which to be fair, you still recognize what milk is and all of that. The machine's not doing that for you. Yeah.

No, it is. So, I mean, yeah, no, I think that's, that's exactly right. And then in terms of the reason why I think it's that, that why the reason why the milk issue came up is specifically around thinking about how machines can make themselves smarter and they need access, they need certain permissions to do things in the world. Maybe they don't actually need all of those actual effectors. They don't need to like be able to identify what milk is at the same time as like being able to like find the best price for, you know, for it and so on and so forth. Maybe you could be— you're solving part of that task and then outsourcing the rest to a human effector, essentially. And in the sense that if you're like, well, go find the best price on these types of GPUs and assemble them in this way so that, you know, you can make, you know, more and more and more of yourself, you know, with the right kind of funding, you can imagine, you know, the kind of feedback loops that are, you know, leading to, you know, what has been called the singularity or something like that, you know, vast, like exponential growth in the improvement of the machine improving itself, maybe not as far away as we think. So, which brings me to the question, this is the question. So when is it going to be? When are we going to—

You're really pushing, you're really pushing for this.

When are we going to have artificial general intelligence? This— I want a year. I want a year. You want a year? I want a year.

I'm going to say 20 years into the future because that's what everyone's always said, right?

I was coming— I was close to that. I was going to say 2050, which is like 28 years, but I was going to say 2050 also.

I was going to say 2050 also.

Well, if we're doing Price is Right rules, then I'll do, uh, 2025.

The thing is, like, even— it's funny, part of me, and I guess I need to think more about or imagine for myself more what this world looks like, is like, I see a world where many of the ways that we're defining AGI are achieved in that period of time, maybe. But I'm still not bought into the singularity. And so then I need to figure out what is that world where we have AGI, but there is there's no singularity. Right? There's like things I think that are like closer, right? Like I was thinking of an example, like playing off of your milk example, right? Of like think something that like I could see you doing with a language model pretty soon, plus some other models. You know, there's already like smart fridges that like probably have like a camera inside of them and try to like see what is in the fridge and remind you of like it knows what's in there and what's not. And imagine you like, you know, you feed the output of that in a language-based format to a large language model. So it's just like streaming information to it about what, what is in, what is in your fridge and has access to information and has the ability to order things for you on Amazon, which obviously the fridges probably already do, but it can be much smarter.

It has been around forever.

Yeah, yeah, they've been around forever. But it can be much smarter about it, right? It can think about what does this mean for like the types of food these people like to make? And what are some other things I could order for them that would be like—

make some suggestions about what they should eat.

And yeah, make some suggestions what they should eat. And then I can order it for them on Amazon automatically, and it will come to their home, or Instacart or whatever.

And then someone needs to cook it. So like, you know, that's, that's, that's where the human is still way ahead, right? Like, the machines are not so good at cooking novel recipes.

No, for sure. Even like there was like that burger flipping robot that was like very popular in San Francisco briefly. Yeah, we went there. That was—

yeah, but it was not very smart. It did not look like a smart machine.

No, for sure. And so there's like parts of it though that you can start to see if you stitch these different— a few different types of models. And actually, that's an interesting point too, is like you have a model over here that I don't know, knows what's in your fridge and keeps track of things. And then you have another model over here that's a language model that's like doing other things and ordering things for you, and you connect those things to each other. And so the other thing to keep in mind is there's, there's literally the connection of like the, the DALL-E type example where these things are actually like deeply connected to each other through layers of weights. And then there's a version where it's, no, these are actually a bunch of components that can speak to each other., and we can almost imagine that you can either imagine them as being parts of a brain, you can imagine them as being a society, like there's different ways to think, you know, to analogize them. Um, but, uh, even if there are society, is that society like hyper-intelligent, uh, globally when you think of it together, when you think of all the parts of it together? Um, but yeah, I mean, I, it starts to make you think you could get there by 2050, make me think that. Yeah, I, yeah, yeah.

I mean, I guess in that world, it's clear why the singularity is not so imminent in that world, because you do have to stitch together different pieces and it requires physical resources. And those physical resources will be constraining in certain ways that are not infinitely exponential, that are not just going to this absolute pulse. Of progress, but are, you know, yeah, fast, fast.

Yeah, you don't have to—

infinitely fast.

You'd have to get to a place, right, in that world where like those machines are talking to each other in a way where they are generating novel, novel tasks to solve, novel things to do, but like from machine to machine, which, you know, is possible.

I mean, possible. I mean, that's entirely going to happen.

Just don't hook them up in a way that they have control over their own power or something like that. There's the person who's going to go and say, and I'm half joking here.

Don't give them access to the network.

Yeah, exactly. There always should be a person who can turn the thing off and they have no way of stopping you from turning it off.

Okay, so another thing that gets brought up a lot in, you know, thinking about bad things that can happen with superintelligence is people talk a lot about motivations. And it's, it's hard for me to imagine current, current models having motivations in the same way that I think of humans having motivation. So the kind of motivation that might make me deceive you to achieve a greater end, right? And that's, that's what a lot of people worry about, that we're going to get some robot that, or, you know, some AI.

You don't, you don't need that though, because you just set up the, the loss function in the right way, right? So like the what function? The, you know, just you set up the rules that this thing is learning by in the right way so that, you know, you are asking it to solve a certain kind of problem. So For example, we talked about, there was that New York Times article that I sent the other day, Rolf, about Diplomacy, that game that the AI won this tournament, this Diplomacy tournament, which is like a World War I simulation. It's like chat-based. And so people are talking, everyone thought this was a person that won this tournament. It was an AI. So it was passing the Turing test in that limited context. But in that case, that's all about deception. And that all the only motivation that machine needed was to say, like, win this game.

Like, we're telling you the only motivation there is that is the goal state. Exactly.

Yeah. Yeah. And so, yeah, you can definitely—

and this comes up when you start having machines teaching each other. Like, if you're saying make yourself better, you know, like, that's— you could see how that could go wrong really, really quickly.

Yeah. Um, On the motivation side, yeah, I agree, Joe. I think you can basically— their primary motivation, the primary motivation of the GPT-3.5 model underlying ChatGPT is to do a good job of providing the output that is required. That can take any form, whether that be lying, whether that be Whether that be being really good. In fact, right, one of the things that, you know, a lot of, there's this topic, the heavy topic in AI that you'll read about around AI safety and there's specific, you know, OpenAI is thinking about this, but I know there are other companies out there who are really focused on this idea of like steerable AI, which basically means, can we make it do things like steer it toward the outcomes desired by humans and desirable to humans. And so it's— and, you know, literally you will like see people add prompts to some for like GPT models that are like, you know, you are a good and honest AI who likes to, you know, who wants to provide helpful answers to humans and like prompt it with that so that it like takes a tone and approach.

Give it some moral nurturing.

Exactly, exactly. And then they, I assume in the tasks, they do things with the tasks themselves to try to like orient it toward that. The, when I say tasks, sorry, the training, training it to solve specific tasks that is also oriented toward that is my understanding. So it's definitely, but I hear, I know what you mean. Like it doesn't have its You know, you can't say like it has its own motivations separate from what it is being asked to do or what it's being trained to do.

Yeah, because we feel— I mean, it gets discussed a lot as motivation, but motivation feels like something maybe less on the surface and more embedded within the system, I guess.

Yeah. And I would also say like this is a whole nother podcast because this is an interesting question, Ralph. It's a whole nother podcast because, I mean, I think you could say something— you have a similar conversation about human motivation. Yeah. And like, what is that really like in that way? Is that really what we think about when we think about motivation?

Yeah, that's what I love about a lot of this is I think, I think you can always go back to, you know, what does it say about us? Why isn't an AI conscious or something? You could say, well, why are, why are we conscious? I mean, you gotta you have to— yeah, figure that out too.

Can I make one really meta point then? This goes back to philosophy. This is more on the philosophy side. I, you know, there's a— often when people talk about like our models of human cognition and human intelligence, we often talk about how they are often based on analogies to the machinery and like of the, of the era in which we're talking about. So if you go back really far, you know, how does the brain work? How does the mind work? You go back further and there's like people talk about like fluid systems because of like, you know, plumbing and water and things like that. We get electricity and like electrical systems in the '70s, you know, computers are becoming more common in '60s and '70s in universities. You get the cognitive revolution, you know, as information processing machines because, you know, that's what we were starting to use. And so it's interesting now. It's very weird and meta. Is these like AI systems are making us re-analogize and think about human cognition in different ways, right? All of those surprising aspects are also like making us think about, well, actually, how much is based on like, you know, a big thing that's been surprising to me is the ability to really change the way these things work and get them to do different tasks just through a prompt versus in addition to what the huge amounts of training they got. Yeah. And like, that's been really interesting because the prompt can so radically seem to change their motivations and how they act, etc. And that's been, that's been surprising to me. And so I think it is like this technology is going to push our assumptions and change our assumptions about like what is intelligence, what is computation. I think a lot of the— or what is, what is cognition? I think a lot of the debate, the early stages of the debate that are really, that's really just starting, you know, started really, I would say, last year that you're seeing just on Twitter and wherever, is these early stages of like the folk psychology of intelligence and cognition is being deeply affected by the surprising aspects of these models. And, and I just think it's, I think it's, you know, I saw Joe, you'd written in some notes for today, uh, something around the family resemblances and Wittgenstein, and it's exactly that. It's a Every time we get some new technology, we learn that our assumptions and theory and folk theories about cognition and intelligence were impoverished, actually. And this new technology is causing us to go, wait, these things I was thinking you needed, you don't actually need. I don't need to assume all of these things. Yeah.

Now this is like a whole season worth of podcast. Yeah, I know. I got to give credit where credit is due.

Rolf actually put the Wittgenstein in there, but I knew that you would appreciate it. As we talked about.

Thanks, Ralph. Thanks, Ralph. All right.

I think this is a great place to wrap it actually, because like Ralph said there, I mean, this is, could be another season. We definitely need to come back and do another one with you though, Daniel Sternberg, because— Absolutely. I'm in. This is fun. Awesome. We definitely, we need to get into this sentient conversation because that was the other way we could have gone today. And I'm glad we went this way because I think it was more generative. But now that we've got this base, I think we've got an opportunity to go there. Which is great. So I'll leave it with one last question, which is we like to ask this when we have experts on the show, which is what are you really excited about now? Like in this, it could be in this field or adjacent related technology stuff. Like what do you, what's coming down the pike that you're really excited about that you see coming?

Yeah. And I'm a little biased because I'm starting to pay more attention to this at work as well. You know, one of the things I mentioned earlier is that you can do so many things with these models once you give them access to more tools to work with. Um, and, and I think over the next year or two, a lot of what you're gonna see in terms of things that are really gonna surprise you are going to come from the ways that these building blocks get used together in to combined with more old-fashioned parts of software, pieces of software that they can interact with. I think you're going to see some really interesting and cool, very generalizable and generalized feeling use cases of these models to solve very specific problems that look very much unlike just like giving you language output. There is no— if they can do, you know, I'll put it this way, you can already get them to write code. You can already get them to generate structured text for you. So a really simple example would be like, you know, Markdown is like the most obviously easy one. Like they can write, you can ask them to provide their output in Markdown so you can format everything really nicely. But you can do way more than that. Imagine they can write SQL, right? Then they can run it against the database. That's a pretty cool example. These things are not far away. They are very close and it's literally just stitching software together. There's going to be, with maybe a little bit of extra training. I think that's going to be really interesting in the near term. I am very excited about the next couple generations of the models as well and what they're going to be able to do. But I think the thing that's going to surprise the most people in the world in the next year or two is going to be, just what happens when you give them more interesting, you know, still inside of a computer, but more interesting effectors to work with.

Great. That's all. That's a great place to wrap. Thanks, Daniel Sternberg, for being on the show.

Really enjoyed it and look forward to having you back. Yeah, thanks for having me.