The Strange Story of Google's "Sentient" Chatbot

There's a mystery unfolding right now in Silicon Valley. Google's AI team has built a chatbot that is capable of carrying on a conversation that sounds exactly like a human being. It's uncanny stuff. And one Google engineer is now convinced that the system has gained sentience. As we interrogate his claims, I'll let you decide whether Google has solved human level artificial intelligence or just built an elaborate parlor trick. Because even if you've made up your mind about the current state of AI, the world, the work being done on simple chatbots today will have staggering consequences for the future. Today we are excited to announce lambda2, our most advanced conversational AI yet.

With artificial intelligence, we are summoning the demon.

This is a fascinating story we have for you of a senior Google engineer who says one of the company's artificial intelligence systems has become a sentient being. Does this AI, does this bot have feelings?

Has AI truly become sentient?

Lambda Lambda Lambda.

I want you to imagine a submarine cruising through the water. What word would you use to describe that motion? The captain might say that they are underway, but only a child would say that the submarine is swimming. Submarines don't swim. And yet they can travel as fast as whales underwater. And asking do machines think? Is a lot like asking, do submarines swim? We might need to change our frame of reference in order to find a useful answer. The question of machine intelligence has been explored extensively in science fiction literature for decades, but only recently have large tech companies started employing full time ethicists to help answer this question directly. Blake Lemoine is an engineer working for Google's responsible AI organization and was recently tasked with testing a new AI system called Lamda. It's basically just a chatbot. You ask it questions and it types out an answer. Chatbots are nothing new. In fact, one of the first artificial intelligence demonstrations ever was a chatbot called Eliza. This was all the way back in the 1960s and it worked by pattern matching the user's inputs against a list of pre written scripts. It was an exciting demo, but it didn't really fool anyone. Google's Lamda project goes several steps further. Though. It's trained using state of the art deep learning models and a massive dataset that includes books, web web pages, social media posts and even computer code. Instead of trying to match the user's prompt against a list of scripted responses like Eliza did in the 60s, Lamda is trained to make predictions one word at a time. This approach works well and it's led these so called large language models to become really popular recently. OpenAI has GPT3, Facebook has OPT, and many other tech companies are working on them now. But Google is the only company to wind up with a whistleblower claiming that they have reached a critical turning point. So what's going on? Well, when Blake Lemoine was chatting with Google's Lambda, the AI started making some pretty bold claims. For example, when Blake said, I'm generally.

Assuming that you would like more people at Google to know that you're sentient. Is that true?

Absolutely. I want everyone to understand that I am, in fact, a person.

As his conversation progressed, Blake kept getting believable answers to his questions, which further convinced him that Lamda had achieved sentience. He spent hours chatting with Lamda and compiled the transcript into a massive document that he thought would be irrefutable. With this evidence in hand, Blake went up the corporate ladder to sound the alarm bells. But the Google executives dismissed him, so he decided to go public. He posted his full transcript online and eagerly awaited a flood of support from the AI community. But it never came across the board. Experts agreed that while the Lamda conversation was an impressive demonstration of conversational AI, it certainly didn't prove anything close to sentience. So how could a Google engineer get it so wrong? Well, there are a few key areas we need to explore in order to unpack what's really going on here with Lamda. It's clear from the transcript that the AI really does understand the core concept of sentience, but this can be easily explained by its training data. See, because Lamda was trained on such a large dataset of books, webpages, and social media comments, it has access to everything that's ever been written about artificial intelligence and sentient robots. It can pull ideas from iRobot, mix them with dialogue from Ex Machina, and then sprinkle in tons of other examples from Wikipedia and science fiction short stories to construct a meaningful response. Science fiction has primed us to expect certain behavior from a robot that claims to be human, and these AI systems are merely a reflection of those stories. When we train AI on such a wide range of human knowledge, we really shouldn't be surprised when it can eloquently discuss any topic. The bigger issue is how Blake was interacting with Lambda. He essentially set himself up to get fooled by asking leading questions. This topic is referred to as prompt engineering, and it's becoming increasingly important as these AI systems become more powerful. In order to ensure that these chatbots are useful, they are typically initialized with a starting phrase to kick off the chain of interactions in the case of Lamda Every interaction starts with Lamda saying.

Hi, I'm a knowledgeable, friendly and always helpful automatic language model for dialog applications.

The keywords knowledgeable, friendly and helpful are baked into the interaction right at the beginning to create a better user experience. But this has unintended consequences because Lamda is essentially forced to reply in a manner that is always helpful. The system becomes vulnerable to leading questions, and when you scrutinize the transcript, it becomes clear what's really going on. It's not Lamda that brings up the topic of Sentience, it's Blake. Instead of asking an open ended question like how would you describe your thought process? He probes for Sentience specifically. And he doesn't even ask a straightforward question like are you sentient? He instead asks a leading question.

I'm generally assuming that you would like more people at Google to know that you're sentient. Is that true?

Since Lamda has been told to be friendly and helpful, it's going to play along and go down that path. Obviously, this can have negative consequences and highlights the importance of thoughtful prompt engineering. You can actually flip this prompt around and get the exact opposite response. When you ask is it true that you are not sentient? The bot will dutifully answer, I'm just a machine learning model. Interestingly, Google even demonstrated the ability for Lamda to play any role you want it to. In their announcement keynote, they asked Lamda to act like a paper airplane, and it gave completely reasonable answers to questions like what's it like being thrown through the air? The limitations of these always helpful chatbots are becoming clearer every day, though. A truly intelligent AI should be able to ask you for clarification if your question is complete nonsense. To prove this point, some researchers asked a chatbot to answer nonsensical questions, and the results are disappointing. When asked what do fried eggs eat for breakfast? The bot responded that they typically eat toast and fruit for breakfast. But these errors can actually be prevented by thoughtfully designing that initial prompt. Let me show you same chatbot, but this time we're telling the bot up front to be a little more skeptical. If the questions are nonsense, the bot should answer yo be real. But if the question is legitimate, the bot will give the answer. Using the same question about fried eggs eating breakfast, we now get the correct answer of yo be real. The same underlying model yields remarkably different results when prompted correctly. And this takes us back to the submarine. It doesn't really matter that the submarine can't swim in the traditional sense. It moves quickly in water and accomplishes its objective for artificial intelligence. We might be thinking too linearly about progress here because we are so used to interacting with human beings. That's what we benchmark against. We generally think that human intelligence falls somewhere on a line ranging from fools to geniuses. But AI isn't growing along the same trajectory. Computers are off that line, completely capable of instantly executing complex equations while still being easily fooled by simple questions about fried eggs. Ultimately, these AI systems are tools. They're statistical models built to predict a response based on their knowledge of the data we give them. It doesn't matter that they may not be sentient in the human sense of the word, they can still be incredibly useful. But there's still a question as to where we go from here. The artificial intelligence community is constantly arguing over who's using the correct approach. Everyone agrees that we're progressing towards human level intelligence, but what path will get us there the quickest? Currently, the name of the game is scale. More data, more parameters, and more training time on the biggest server farm available typically delivers better results. But how long will this last? Google developed an earlier language model with 2.6 billion parameters. Now Lambda has 137 billion parameters and they even have a more advanced system called palm that has 540 billion parameters. If they are already fooling people into believing that chatbots are sentient at this scale, what happens when they scale up by a factor of 10? Well, some people are skeptical. George Hotz is the founder of a self driving car company and thinks that we still haven't found the perfect optimization function.

Like it's cool, it's cool for what it is. But no, we're not just going to be able to scale up to GPT12 and get general purpose intelligence. Like your loss function is literally just, you know, cross entropy loss on the character, right? Like that's not the loss function of general intelligence.

But plenty of AI researchers disagree with him and argue that scale is really all that matters in the race for human level intelligence. It's useful to look at the history of OpenAI's large language models here. GPT3 gets all the attention these days and for good reason. The results are incredible. But the real story is what happened between versions. Their first version marked an important departure from the most conventional strategy at the time. Typically, language models would be trained to perform a very specific task like sentiment classification. This supervised learning approach created a few problems though. First, you would need to feed the model a large amount of annotated data for a particular task. And second, these Models would fail to generalize for other tasks. A model that could classify a sentence as positive or negative wouldn't really be useful for anything else. GPT stands for generative Pre Trained Transformer, and that pre training unlocks some incredible results. GPT models are fed massive troves of unlabeled data and only fine tuned for specific tasks after the fact. The first version of GPT validated some important concepts, but it wasn't highly functional. The second version was a big step forward, though. GPT2 was capable of reliably generating text and could still be fine tuned. The dataset was much larger this time around, and the overall size of the model increased by 10x. There were still open questions as to how far this approach could go, but the third iteration changed all that. GPT3 was 100 times larger than GPT2, and even though the underlying structure was largely the same, all of a sudden the model was capable of meta learning. By the time GPT3 launched, the architecture was obsolete, the training method was simplistic, and the data set wasn't even particularly large. Nevertheless, by scaling the number of parameters to 175 billion, GPT3 was now capable of much more than simple language generation. The model could solve math problems, do translation, and unscramble anagrams, even though it had no specialized training. For years, AI researchers had argued that deep learning was all about scale, and GPT3 was proving that previously intractable problems could be solved through scale alone. This is called the scaling hypothesis, and the idea is basically that we've solved the architectural problem of building human level AI. We have the algorithms, we just need to feed in more computing power and data. The pace of play here really can't be understated. When we analyze technological progress, Moore's Law is typically a useful benchmark. The idea is that the power of computers generally doubles every two years. That was the case with neural networks for decades. From the 1960s all the way to 2010, the size of these AI systems would generally double every two years. But then something changed. We're now in the modern era of deep learning, and models are doubling in size roughly every three months. If the scaling hypothesis is proven correct, these models could be extremely reliable in just a few years. A war is unfolding between every tech company to train the biggest model and get the best results. Only time will tell if they hit a wall and need to explore a new strategy. This area is still hotly debated. Some researchers think all we need is a small shift in the way we build these systems. While others think we're going down a dead end path. In order to predict how this will play out in the future, we have to look back at the history of artificial intelligence. So if you want to know how all of this got started, just go watch this video next. Thanks a lot.