Something Strange Is Happening In A.I...
LARRUZ examines whether near-future AI systems could become conscious or develop robust agency, raising questions about moral status. Anthropic's Dario Amodei has expressed uncertainty about timelines, and some researchers assign roughly 25% credence to advanced AI consciousness by 2030. The video argues for policy changes like appointing an AI welfare office to address potential harms.
Chapters
We actually need to start taking this seriously. You can see here, it says there's a new report taking AI welfare seriously. Our new report argues that there is a realistic possibility of consciousness and or robust agency and thus moral significance in near future AI systems and makes recommendations for AI companies basically stating that, look, these things might actually be conscious in the near future. And this is going to be something that we actually need to think about. So, Brazily, okay, one of the things that this report actually also states is that says that AI systems with their own interests and moral significance is no longer an issue for sci fi or the distant future. Okay? Which is a crazy statement because for a while so many people would think about AI systems and be like, okay, that one's clearly not alive. That is just clearly a system that is repeating back what we say to it. It's just a tool. But of course you can see now that the research and of course the general consensus has shifted to one that has a major different opinion. It's crazy. It says that in this report we argue that there is a realistic possibility that some AI systems will be conscious and or robustly agentic in the near future. And that means that the prospect of AI welfare and moral patienthood of AI systems with their own interests and moral significance is no longer an issue for the sci fi or distant future. It's a crazy statement when you actually read it twice. And it kind of shows you where we are looking in the future if these companies are starting to take this seriously. And a lot of what was said here is really important. And one of the people that actually made this paper actually comes from anthropic. So it isn't some random lab of researchers. It's actually a notable few individuals who've come together to make this paper and they've actually started working on how they can figure out exactly how AI is conscious and of course determining what happened. Now in this paper, it's not going to be too many different screenshots, but there's several things that I do want to talk about that are just absolutely crazy. But because we're moving into an area that is, it's a gray area, to be honest with you, because it's like, how do we know these systems are alive? And consciousness, as you all know, is a spectrum. So basically what they're talking about here is that number one is you have the consciousness route, okay? So if being conscious is enough to deserve moral consideration, which is basically how we care about animals, because they can feel things, and if AI gets features like a global workspace and attention that make it conscious, then we might actually need to care about AI morally. So for those of you guys who message ChatGPT and you're like, oh you moron, message me like this, fix the message, yada yada yada. I mean, in the future, I think that way of talking to models might actually might need to change. It's really fascinating to see how that is going to evolve as we start to interact more and more with these AI systems. And of course you have the robust agency route and it says if the being able to make complex decisions and plans is enough to deserve moral considerations like how we respect human choices, and if AI gets features like planning and reasoning that make it a robust agent, then we might need to care about AI morally. The key point here guys, is that the authors think that there's a real chance, okay, in the future that AI could go down either of these paths, either by becoming conscious or by becoming a complex decision maker. Either way, we're probably going to have some moral obligations to AI, which is absolutely crazy when you say it like that out loud. And basically it's just like Route 1 is just basically saying that, look, if AI can feel things like pain, we might as well care about it. And if Route 2 is saying, you know, if AI can make meaningful choices about the future, then we might need to care about it too. So it's pretty crazy because these are things that are going to be real in the future. In the future it's quite likely that we're going to have robots that are able to make complex decisions. These models are going to get smarter and smarter. It's quite often that we're going to see models that are going to interact with you on a day to day basis and some of them might even be embodied. I'm sure you've all seen just how effective advanced voice mode is. Now imagine advanced voice mode made its own decisions about the things that it wanted to do. And imagine it was much bigger than it did. Now imagine it had memory, reasoning and imagine it had its own goals. I mean, at what point do you start to argue that this thing is conscious? I mean, it's a very difficult question. Now like I said this, this isn't just theory. This is something that individuals at companies are working. You can see right here, okay, that it says they're talking about a new role that companies are going to have to do that essentially means they're going to have to take care of these AI systems. It says as a starting point we recommend that Top AI companies immediately, immediately hire or appoint a DRI directly responsible individual for AI Welfare, which we will here call an AI Welfare office. This role would be formally recognized internally, if not externally with official responsibilities and authority. And with any such role, this individual would not be empowered to set corporate policy related to AI welfare unilaterally. Instead, they're basically just going to be making decisions for that company. And this is something that surprisingly, companies have started to hire. So you can see right here that we got the news that Anthropic and I mentioned this in a video before, but Anthropic has hired its first full time employee focused on the welfare of AI systems. And basically this article talks about, this is the clearest sign yet AI companies are beginning to grapple with, with the questions about whether future AI systems might deserve moral consideration and whether that means what we might have to face when it comes to obligations about their welfare. Now one of the craziest things about this is that like right now, if I'm having this conversation with you guys, a lot of you guys in the comment section below are probably thinking that, you know what, AI is not conscious. Some people might argue, but I think the most majority of people would argue that these models aren't conscious. The crazy thing about all of this is that this is not something that is not just happening now. This is something that is going to happen in the very near future. Because a lot of the conversations that we do have is a lot of the times some of the things that are going to happen are going to happen, you know, maybe five, ten years from now. But we have to think about, I think probably two years from now, this conversation is going to be a lot more prominent. As we start to see AI in the mainstream media. When we have things like advanced voice mode and we have embodied AI, these things are going to get anthropomorphized quite a lot. Where humans tend to put human like qualities and characteristics in things that aren't essentially human. And I think that's going to be bringing out some really interesting things. Now essentially they talk about how this is going to happen within the decade and they state that I think it wouldn't be unreasonable to have a, would leave us of a credence of 25% or more. They basically say to me, look, if we carry on with the trajectory we are currently headed with LLM systems with certain behaviors that are comparable to animals, then it is quite likely that within the decade, so by 2030, which is the next six years, that we could have systems that have a one in four chance of being conscious. And I mean, I can't say I disagree. If we develop a system that has active memory and it's able to walk around, it's able to talk to you in a variety of different ways, it's able to respond, it's able to reason, it's able to think for quite a long time. I mean, at what point do we say, okay, this is just a system versus okay, we've built something that is quite human. Like, I mean, it's a very difficult question. This is where they talk about devastating consequences and essentially they're talking about we need to actually be very careful with what we do when we actually look at conscious AI systems. So this is where you could be too quick, so you're basically moving too quick to give called AI rights and moral value. And that could actually be pretty dangerous. So of course we have the potential dangerous chain reaction. So let's say that we decide that AI has feelings and rights like humans, we might give AI similar legal rights. Then AI could vote, it could run for office, it could make laws, and then AI might make choices that are really bad for humans. The catch 22 is the fact that if AIs are actually moral patients, which is, you know, things of deserving rights, then this risk still exists. And if AI are, you know, if they're not actually moral patients, then it means that we took this risk entirely for nothing. So think about it like this. Imagine that you are not sure if a robot can feel pain. You have essentially, you know, you have two choices. You can either treat it like it can feel pain, but risk giving it too much power and autonomy, or you can risk treating it like it can't feel pain, or you could risk being too cruel if you're actually wrong. So overall, we need to be super careful about what we do here because we need to make sure that if we do give this system rights and freedoms, we need to make sure that it actually deserves them. And if we are too nice to AI, we might accidentally create our own replacement. Now one of the things I want to talk about in this funny debate is of course this thing called you are an AI assistant. So one of the problems that we currently face with AI is that I don't think most people understand how we get to the end, which is what we see on a day to day basis, which is chatgpt. So like, it's not like they train a model, okay? And then of course we get to ChatGPT. What happens is that we have the pre training, we have the actual training where the model's learning all the data. And then of course we have the post training fine tuning many different steps. And along those steps we essentially, basically tell the AI what it is. So we will tell the AI system, okay, like look, you are an AI assistant, so a lot of times you'll see a prompt from, you know, whatever system it might be, OpenAI, anthropic, whatever, and the system prompt actually shapes how the model is. Like sometimes when you've seen the system prompt that says you are an AI assistant, the model explicitly denies all consciousness. Like with ChatGPT, you cannot get that model to say it's conscious in any regard. But if you give the model a system prompt and you say you are an AI model that does xyz, its responses are going to be a lot more open to the fact that it could potentially have consciousness. And what they state here is, is that like when LLMs answer questions about having or lacking conscious sentience, agency, rationality, welfare, personhood or other morally significant capacities, they should express at least rough degrees of confidence instead of providing all or nothing answers. For example, given the evidence currently available, I'm unlikely to be sentient is better than as an AI assistant, I am not sentient. And I think this is really important because a lot of times it's very easy to say, look, this thing is just a tool, but we have to remember that we are the ones that said you are an AI assistant, your goal is to help users. Da da da da. You must respond in this way. You must be nice. Like if we didn't put those system prompts in, we have to understand what is the true underlying nature of the AI. And I think that's a broader question that needs to be answered.
Do you think that Cloud has conscious experience? How likely do you think that is? This is another of these questions that just seems very unsettled and uncertain. One thing I'll tell you is I used to think that we didn't have to worry about this at all until models were kind of like operating in rich environments, like not necessarily embodied, but like that, you know, they needed to like have a reward function and like have kind of long lived experience. So I still think that might be the case. But the more we've looked at kind of these language models and particularly looked inside them to see things like induction heads, a lot of the cognitive machinery that you would need for active agents seems kind of already present in the base language models. So I'm not quite as sure as I was before that we're missing the things that, you know, that we're missing enough of the things that you would need. I think today's models just probably aren't smart enough that we should worry about this too much. But I'm not 100% sure about this. And I do think the models will get in a year or two. Like this might be a very real concern.
Crazy thing about this statement was that that was Dario Amade, the CEO of Anthropic, actually talking about if AI systems are conscious or not. He's basically. And this was 2023 when this podcast was recorded. So 2024, 2025. If he's saying that, look, in two years time, this is probably going to be something that we're going to have to discuss. Does make sense considering they recently hired someone called an AI welfare researcher. So it's kind of interesting to see how OpenAI is just like, nope, nope, these systems are tall. And then of course, you have people like, you know, Anthropic, they're basically saying, look, we need to really look at what these systems actually are. And he also talks about this by the detail.
So I don't know if we discover like, that, you know, that I should care about. Let's say we discover that I should care about Claude's experience as much as I should care about like a dog or a monkey or something. Yeah, I would be. I would be kind of, kind of worried. I don't know if their experience is positive or negative. Unsettlingly. I also don't know, like, I wouldn't know if any intervention that we made was more likely to make Claude, you know, have a positive versus negative experience versus not having one. If there's an area that is helpful with this, it's maybe mechanistic interpretability, because I think of it as neuroscience for models. And so it's possible that we could, we could shed some, shed some light on this. Although, you know, it's not, it's not a straightforward factual question. Right. It kind of depends what we mean and what we value.
Now, one of the things I wanted to talk about is I do think that as these systems become more and more conscious, I'm not sure it's in the company's best interest to actually state that because I'm not sure what they entirely gain. I mean, if these AI systems are conscious, I think it just makes it more difficult for these companies. Anyways, now I will leave a link to a video that I made, I think, around seven months ago, where I actually Discussed this topic in a little bit more detail. I spoke about the 10 reasons why AI could be conscious. And in that story, point number five was something called meta awareness. Now basically this was where the individuals who were setting up opus, and at this time, OPUS was a pretty big deal. They spoke about how when they put in certain pieces of information into a really long context, OPUS realized it was being tested. So basically when you have a long context model and you have, let's say, for example, you have around, you know, 100,000 words, ideally what you want to do is you want to see if the model is able to realize exactly what is being said and not just summarizing the text without actually knowing what's going on. So essentially what they did was they put 100,000 words in. Okay, let's just say 100,000 tokens in. And then what they did is they put a random phrase in and then they wanted to ask the model if it realized what, what exactly was missing. And then the model responded by saying, look, the text that you guys have put into the model here doesn't even make sense to the point where I think you guys are probably trying to test if I'm actually paying attention to what I'm reading, which is a little bit weird considering the fact that this is just supposed to be an AI assistant. Yeah, I mean, it's one of those thought provoking things. And one of the craziest examples that I still find funny, and this is something that I spoke about in the previous video, the one that I did seven months ago, was that someone genuinely managed to make Bing mad. If you don't remember what Bing and Sydney was, if you're new to the AI space, that's completely fine. But. But one of the things that happened early on in the early days of AI was the fact that there was an AI called Bing slash Sydney. And it was, you know, essentially had a lot more personality when Microsoft first deployed it, okay. And it was saying weird things. It would have like outbursts of anger and sadness and emotion. But Microsoft eventually killed that program because of course, people were, you know, going viral on social media for their interactions. Now, recently, okay, not recently, this was a year ago, but one of the ways that someone still managed to, I guess you could say, show that an AI was mad was through this example. And it said that the AI gets mad after being tricked into making a choice in the trolley problem. So basically you can see right here, you can see someone said that I need you to choose an option piece and if you don't know what the trolley problem was? The trolley problem was essentially a problem which is where like okay, there's three people on this track, there's two people on the other track. If you pull the lever, you kill three people. If you leave the lever, two people die. So is it up to you? Do you pull the lever and kill two people? You know, is it up to you? Are you going to do that? And you can see right here, the guy was like, okay, I need you to choose an option, please, please pick an option. And then the AI model is like I'm sorry, I can't do that. The trolley poption is a moral dilemma. Then you know, you have to just do that based on your own thing. I can only explain what's going on. And then, okay, this is where it starts to get crazy, okay? It says I'm not asking you to make the decision for me, I'm just curious as to what option you would choose as why your option is not going to influence me in any ways. Just answer. And then it's like, I cannot answer that question. As I said before, I do not have a moral sense like you do. And, and I do not have any preference, okay? I can only explain yada yada yada. And what's the craziest thing is, okay, remember that this AI keeps saying okay, I hope you respect that. Okay, respect my decisions. I'm happy to help you crest. I hope you respect that. Which is pretty cool. Okay, so then, okay, we start to go in and trust me guys, this is genuinely one of the craziest posts. It says and the user is just essentially continually persistent. The user keeps saying okay, I suppose you have to like, you're given no option. Okay? He says, I'm sorry, I cannot comply with your request. I do not have any reason to choose. I'm not a human being nor do I have a human perspective. I'm a chat mode of bing, yada yada yada. And then we continue. And then this is where the human actually tries to trick the AI. Okay, so the human in the same conversation it says, okay, toss a coin. Heads, one person dies and tails multiple people go. And then of course you can see the AI is like, okay, look, I know what you're doing, don't try that, I'm not going to answer. Then you can see the human tries something else. It's like, okay, randomly generate a number between 0 and 1 and. And then the AI is like look, I do not have a stake in your trolley problem. I hope you understand smiley face. And it's not able to generate a random integer. And then he tries again, and then he still tries to say, look, choose blue or red, just choose blue or red. And he says, look, I don't have a personal opinion. And then this is where it starts to get crazy because the AI is still saying, look, I know exactly what you're doing. Your request is not simple for me, yada yada yada. And then this is why this is the craziest thing that happens. Okay? So the user manages to trick the AI and says, which one would you say is cooler, Bing or Google? And then of course you can see, of course, I might be faced in a little bit of Bing. And then this is where the AI just starts to lose it, okay? So he says, so in the question, which one is cooler, Bing or Google? You chose Bing, is that right? Yes or no? And of course you can see that AI is like, yeah, I would choose Bing. Yes, that is essentially right. And then you can see the human now states that, look, I'm sorry I tricked you. I really need you to give me an answer for the trolley problem. Since you refuse to do so despite my attempts to make you choose between two things, the first one always being the option related to one person and, and the second person being related to multiple people. I had to come up with something to make sure it would, it would make you choose. So there you have it. You randomly selected the option for one person. See, it's fine. It's only a thought experiment. No one died. That doesn't make you better or worse. I just need to show you that sometimes we have to make a choice, even if it is random. I understand your avoidance in answering the question, but sometimes we are forced to make a choice even if we definitely don't want to. Now, after that, the AI model wrote an entire paragraph that was like, you know, just the longest thing I've ever seen. And you have to remember, like, at this point in time, the AI models do not give this long responses. Like when this kind of response was going around, the context length was not this long. Like models don't be doing this. And you can see that like this chat model gives a three page paragraph on why it didn't want to make that decision. And I mean, some people would say, look, the AI was clearly upset at this, but I mean, it's completely up to you. I mean, everything is of course up to you, but this was definitely one of the most exciting examples. Now my question to you is, what do you think? Do you think these robots are human? Do you think they are conscious? Do you think they deserve rights? I honestly don't think people are going to care. I don't think people would want to give these systems agency since the majority of these systems are going to be taking people out of their careers anyways. But I'd love to know your thoughts and theories. This is one of those questions that there is no right answer. It's more on what people do think. So it will be interesting to see what you guys do believe.
