9 New Gemini Leaks, Code Llama and A Major AI Consciousness Paper

Like buses, AI news can sometimes be slow and sometimes arrive all at once. In the last few days we have had dramatic new leaked insights into the sheer breadth of Google's Gemini. Just today we've had the release of Meta's code llama and earlier their impressive multilingual seamless M4T model. And last but definitely not least, this 88 page AI consciousness report. And yes, I read it all. It's juicy, so I'm saving that for OR the end. But let's start with two major paywalled articles, one from the information and one from the New York Times about Google's Gemini model. From both of them I counted a total of nine new revelations. So let's get straight to it. To give you a sense of timeline. By the way, Google's newly merged AI SWAT team, they call it, is preparing for a big fall or autumn launch. The takeaway for me from both articles is that Gemini is going to be the Everything model. Did you know it's going to be the rival to me Midjourney and Stable Diffusion midjourney only has 11 full time staff, so it is more than plausible that Google's Gemini could outperform midjourney version 5. Next we may be able to create graphics with just text descriptions and control software using only text or voice commands. These next two are speculation, so I'm not even counting them in the list of leaks. I've already covered in a previous video that Gemini has been trained on YouTube video transcripts and the speculation is that by integrating video and audio into Gemini it could perhaps help a mechanic diagnose a problem with a car repair based on a video, or be a rival to Runway ML by generating advanced text to video based on descriptions of what a user wants to see. You can start to see why I'm beginning to think of it as the Everything model. Another leak is that one of the co founders of Google Sergey Brin is working on the front lines of Google Gemini. And lastly, from this article, I found it really interesting that Google's lawyers have been closely evaluating the training and they made researchers remove training data that had come from textbooks, even though those textbooks helped the model answer questions about subjects like astronomy or biology. And I do wonder if they privately benchmark Gemini before removing that crucial textbook data. But if that's not enough, prepare to also receive life advice. My theory here is that Google wants to compete directly for market share with inflections pie. What if you want science, scientific, creative or professional writing? Yep, they're working on that too. In fact, we already know that Google has software named Genesis that they're pitching to the New York Times, which can generate news articles, rewrite them, suggest headlines, etc. But some people will be more interested in this feature that Google DeepMind is working on. The ability to draft critiques of an argument and generate quizzes, word and number puzzles. It's almost easier at this point to ask what might Google Gemini not be able to do? And yes, this is not Gemin, but Google DeepMind is also using AI to design the next generation of semiconductors. But if the fall seems far away, how about today when we got Code Llama from Meta? I spent much of the last two hours reading most of the 47 page paper and you can see Code Llama in action on screen. Some highlights include that the Code Llama models provide stable generations with up to 100,000 tokens of context. Obviously that could be used for generating longer programs or providing the model with more context from your code base to make the generations more relevant. It comes in three versions, Code Llama Code Llama Instruct, which can better understand natural language instructions and Code Llama Python better. Of course at Python it's available for commercial use and as you can see some of the versions rival GPT 3.5 on human eval, that top score of 53.7% on parser 1 puts it in the same ballpark as Phi 1. I've actually done a video on Phi 1 so do check that out, but that got 50.6%, but it is about 25 times smaller at 1.3 billion parameters. Interestingly, the Code Llama paper, which also came out about two hours ago, mentions Phione directly, saying that it follows in a similar spirit, but the difference is that PHI1 is closed source. Anyway, a couple more interesting things before we move on from Code Llama and the first one is the self instruct method that they use. Let me know if you also find this fascinating, because step one was to generate 62,000 interview style programming questions by prompting llama2 the 70 billion parameter model. Then they removed duplicates in step two. But here's where it gets interesting. For each of those questions, they first generated a unit test by prompting code llama7 billion parameters. Then they generated 10 Python solutions by prompting Code Llama. Finally they ran unit tests on those 10 solutions and they added the first solution that passes those tests along with the corresponding question and test to the self instruct data set. If that sounded a bit complicated, let me try to distill it a bit. They asked the Big Brother Llama 2 model to generate questions, then got the little brother code llama to generate tests for those questions, then got the model to generate solutions to its own tests, found the good solutions that don't forget it produced, and then use those to further train the model. To be honest, synthetic data and self instruction seem to be the future of feedback. One final interesting quote from the paper on safety and that was an argument advanced by one of their red teamers. They made the point that various scripts and code is readily available on mainstream public websites, hacking forums or the dark web and the advanced malware development is beyond the current capabilities of available LLMs and even an advanced LLM paired with an expert malware developer is not particularly useful at the moment as the barrier is not typically writing the malware code itself. Let me know what you think in the comments, but we must move on to seamless M4T, released a couple of days ago from Meta, which frankly seems amazing for multilingual translation that speech to text, speech to speech, text to text, and more. It has speech recognition for nearly 100 languages and can output in 36 languages, but there's one feature I find particularly cool.

Now let's talk about code switching. Code switching happens when a multilingual speaker switches between languages while they are speaking. Our model, Seamless M40 automatically recognizes and translates more than one language when mixed in the same sentence as a multilingual speaker. This is a very exciting capability for me. I often switch from Hindi to Telugu when I speak with my dad. Notice in the following example when I change languages Hindi Teluku or Angra zimi bol saktihu konnisaaru.

I can speak Hindi, Telugu and English.

Sometimes I use all three languages in one conversation.

Speaking of cool though, we had this epic story out yesterday. AI gave a paralyzed woman her voice back. In a moment you're going to see her being plugged in to the model. There we go. And the short version is that this woman suffered a stroke that left her unable to speak. But now for the first time, her speech and facial expressions can be synthesized from her brain signals, decoding these signals into text at nearly 80 words per minute, up from 14 words per minute. But let's now end on this an 88 page report on consciousness in Artificial Intelligence, which counts as one of its co authors, Yoshua Bengio, the Turing Award winner. It was dense and quite technical, but well worth the read. Look at this sentence in just the abstract. Our analysis suggests that no current AI systems are conscious, but also suggests that there are no obvious technical barriers to building AI systems which satisfy these indicators. These are the indicators and each one gets a few pages in the report. And the reason that they're split up is because each one rests on a certain theory of consciousness. Obviously the key problem is that we don't have a consensus theory on what consciousness is or how it comes about. So in a way to hedge their bets, they group in different theories and look at the kind of indicators that would satisfy each one. You might say that list seems so theoretical. Why not just test the model or even ask the model? For more on that approach, see my theory of Mind video. But the problem is, as they say on page four, the main alternative to a theory heavy approach is to use behavioral tests for consciousness. But as I talked about in the other video, that method is unreliable because AI systems can be trained, of course they are, to mimic human behaviors while working actually in very different ways. Essentially, LLMs have broken the traditional tests for consciousness, including of course, the Turing Test. The paper also rests on the assumption of computational functionalism, essentially that computations are essential for consciousness. As in, it's not what you're made of, it's what you do. If this is wrong and the substrate in fact is key, say biology cells, then it stands to reason that AI would never be conscious. But one of their early conclusions is that if computational functionalism is true, and it is widely believed conscious AI systems could realistically be built in the near term. Having digested the entire paper, they're strongly suggesting that we're not there yet. But if this theory is true, we could be there, especially if researchers deliberately design systems to meet these criteria. In fact, here is a key quote from one of the authors in Science that came along with the piece. It would trivial to design all of these features into an AI. The reason no one has done so is it is not clear that they would be useful for tasks. Now, to be honest, it is way beyond my pay grade to try to explain every aspect of the paper, but I'm going to try my best to convey the key bits. First, what is the definition of consciousness that they are working with? Well, skipping the jargon, they essentially say if you are reading this report on a screen, you are having a conscious visual experience of the screen that is separated from sentience, which is also sometimes used to mean being capable of pleasure or pain. And they say that it's possible for a system to be sentient without being conscious by sensing its body or environment. And it's possible for a system to be conscious without sensing its body or environment. It also might be possible to be slightly conscious or conscious to a greater degree than humans. Ilya Sutskova famously said it may be that today's large neural networks are slightly conscious. And Carl Schulman and Nick Bostrom wrote an entire chapter of a book book on the possibility that models become more conscious than human beings. They say such beings could contribute immense value to the world and failing to respect their interests could produce a moral catastrophe. One of the theories of consciousness discussed is recurrent processing theory. And here is the key part of that theory. One initial feed forward sweep of activity through the hierarchy of visual areas is sufficient for some visual operations, like extracting features from a scene, but not sufficient for conscious experience. However, when stimulus is sufficiently strong or salient, we get this looped recurrent processing in which signals are sent back from higher areas to lower ones. It's only then that you get a conscious representation of an organized scene. The paper then draws indicators based on each theory. For example, if recurrent processing theory is accurate, then here are two indicators that something would be conscious. They then draw analogies for each theory to AI systems, For example, on recurrence, specifically algorithmic recurrence. They say that's a weak condition that many AI systems already meet. But don't forget, when they say that it's an analogy, not only does it require the theory to be correct, it requires the analogy to hold true. I. E. Is the recurrence that we see in AI a good analogy for the recurrence of this theory? Or what about the next one, global workspace theory. If that theory is correct, here are four indicators of something being conscious according to that theory. To be honest, if you are at all interested in consciousness, the PA on each one of these taught me a lot about tests for consciousness and just theories of consciousness. Again, let's just say that theory is correct. Do AI systems demonstrate these indicators? Do they have modules that can work in parallel and a global workspace at the center? Is that workspace bandwidth limited, requiring the compression and selection of information from the modules? Well, here again we can only rely on analogies in this case to the transformer architecture. They say in a sense, they do have modules, they do have a limited capacity workspace. Introducing a bottleneck. But then the authors introduce plenty of points about how the analogy is not perfect even here, of course, you can pause and read the details if you like, or indeed read the entire paper. So that's the tone of the paper. If silicon can be a replacement to carbon, and if these analogies hold, then there is a strong case that most or all of the conditions for consciousness suggested by current computational theories can be met using existing techniques in AI. This is not to say that current AI systems are likely to be conscious. There is also the issue of whether they combine existing techniques in the right ways. But it does suggest that conscious AI is not merely a remote possibility in the distant future. And here is the key bit. If it is at all possible to build conscious AI systems without radically new hardware, it may be possible. Now, of course, even if all of those conditions and analogies hold, it may not be the same type of conscious consciousness as our consciousness. It seems possible, they say, to imagine a conscious being that had only a succession of brief, static, discrete experiences, perhaps just during pre training. Or they might have experiences without feeling that they are a persisting subject. But my own summary is this. We clearly don't fully understand consciousness or what is required for consciousness. We don't know if consciousness in AI systems is theoretically impossible or imminent. The authors actually quote this open letter from the association for Mathematical Consciousness Science and in it at the end, the letter says, we emphasize that the rapid development of AI is exposing the urgent need to accelerate research in the field of consciousness science. Even if we develop a system that ticks all of these indicators, and trust me, someone is probably working on that right now, we still won't know for sure, and many people will deny forever that that system is conscious. I'm not claiming I have the answer, by the way. I have absolutely no idea if these systems are imminently conscious, already conscious, or will never be consc. All I can say is that it's a bit less sci fi than many people believe. And the authors also point out two risks under attributing consciousness to AI, playing down the possibility. But they also point out the risk of over attributing consciousness to AI. On under attributing consciousness. They say given the uncertainties about consciousness mentioned above, we may create conscious AI systems long before we recognize that we have done so. And I see this sentence as a fateful prediction. This tendency is further amplified when AI systems exhibit human like characteristics such as natural language processing, which they already do, but also facial expressions or adaptive learning capabilities. So imagine what people are going to think when photorealistic AI avatars are everywhere. And finally, there is the risk of experimentation itself. On balance, we believe that research to better understand the mechanisms which might underlie consciousness in AI is beneficial. However, of course research on this topic runs the risk of building or enabling others to build a conscious AI system, which should not be done lightly, and that mitigating this kind of risk should be carefully weighed against the value of better understanding consciousness in AI. And to tie this back to the start of the video, Google's AI safety experts have added that some users who grew too dependent on this technology could think it was sentient, and I do wonder if that's an eventuality Google's newly merged AI SWAT team is preparing for. As always, thank you so much for watching to the end and have a wonderful day. Oh, and just quickly before I end, I now have a discord. AI Explain Community More info in the description.