Anthropic Begins to Unlock the Mystery of LLMs

Today on the AI Daily Brief, Anthropic makes a major breakthrough in interpretability. Before that in the headlines, Nvidia just continues to smash expectations The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, check out the discord linked in our show. Notes. Welcome back to the AI Daily Brief Headlines edition. All the AI headline news you need in around five minutes. We kick off today with an earnings report from Nvidia and surprise, surprise, it just keeps getting bigger, Wall Street Journal writes Nvidia delivered a record quarter and signaled that the AI boom is still going strong, driving its already meteoric stock up above $1,000 a share. Revenue last quarter more than tripled year over year to 26 billion and net profit was up 7x to 14.88 billion. Both were quarterly records and both beat analysts expectations. The stock market obviously love this, with share prices up 6% in pre market trading following the report and a single share surpassing $1,000. One of the things that's also really interesting though is that while currently the big cloud companies like Google, Microsoft and Amazon account for around 45% of Nvidia's data center revenue, they're clearly trying to move to a world where they're not just selling to data centers, but also selling directly to companies. At Dell's big annual event this week, for example, Nvidia and Dell talked about how they were trying to create the AI factories of the future where individual companies had more direct to this sort of capacity, CEO Jensen Huang said. We are poised for our next wave of growth. From Bloomberg, Nvidia emphasized Wednesday that it wants to sell its technology to a wider market, expanding beyond the giant cloud computing providers known as hyperscalers. Huang said that AI is moving to consumer Internet companies, carmakers, biotechnology and health care customers. The large scale deployment of Nvidia chips by Elon Musk's Tesla is one sign of that expansion. It continues to be a battle for Nvidia to keep up with demand, said Huang. Nobody has ever manufactured supercomputers at volume. We're doing the best we can. The one other interesting note as called out by the Verge, was that Nvidia will now make new AI chips every single year, said Huang. I can announce that after Blackwell there's another chip. We're on a one year rhythm. The Verge points out that until now Nvidia produced a new architecture roughly once every two years Ampere in 2020, Hopper in 2022, Blackwell in 2024, but that everything is getting faster now, next up in the headlines, you'll remember that recently Microsoft inked a deal with UAE based G42. Part of why I was so interested in the deal is that it seemed to have been facilitated by the Department of Commerce and reflected geopolitics as much as business considerations. Basically, G42 had been right at the center of the US China tension and the US had been putting a ton of pressure on it to pick a side. Well, pick a side it did, and Microsoft's minority investment of $1.5 billion was part of that picking. Still, it's not without complications, writes Reuters. Microsoft President Brad Smith said the tech company's high profile deal with The UAE backed AI firm G42 could eventually involve the transfer of sophisticated chips and tool, a move that a senior Republican congressman warned could have national security implications, said Michael McCall, the Republican chairman of the Foreign Affairs Committee. Despite the significant national security implications, Congress still has not received a comprehensive briefing from the Executive Branch about this agreement. I am concerned the right guardrails are not in place to protect sensitive US origin technology from Chinese espionage, given the CCP's interest in the UAE. To me this sounds a little bit like if this really was coming from the Department of Commerce, that was obviously a White House facilitated deal that Congress just doesn't have visibility into. Anyway, there's tons more details here, but what's interesting to me continues to be just the geopolitical implications of AI and how quickly they've become an ongoing concern in the world of M and A. I'm wondering if we're not about to see a bit of a wave of consolidation in the AI space. We've had a couple boom years of funding and now we're very naturally in the phase where companies are figuring out if there's enough of a there there to raise a next round or if it makes sense to try to join up with someone bigger. One company seemingly going through that decision making process is Adept, which was valued by investors at more than a billion last year and which, the Information Reports, has held talks recently around a possible sale or strategic partnership with large tech companies. Most notably Meta. Adept is in the much vaunted AI agent space. And of course that means they're dealing with not only intense competition from every angle and every direction, but also the fact that AI agents are still at this point highly theoretical. There is an existing consumer demand to tap into. It's a new behavior and a new category that's being invented on the fly. When it comes to something that challenging, they may decide that it makes sense to do that from within one of the big giants that has the capital to actually pursue it to its full ends. Speaking of big companies, in what will be a surprise to no one, Amazon is apparently planning to give Alexa an AI upgrade as well as a monthly subscription fee. Notably, this will not be included in Amazon prime subscriptions, writes cnbc. Amazon will launch a more conversational version of Alexa later this year, potentially positioning it to better compete with new generative AI powered chatbots from companies including Google and OpenAI. Will the end of the year see us watching a souped up Alexa compete with a souped up Siri, both of them competing against some brand new OpenAI product? Kind of seems like it. Speaking of an OpenAI product, the Washington Post reports that OpenAI didn't actually copy Scarlett Johansson's voice, they write. When OpenAI issued a casting call last May for a secret project to endow OpenAI's popular ChatGPT with a human voice, the flyer had several requests. The actors should be non union, they should sound between 25 and 45 years old, and their voices should be warm, engaging and charismatic. One thing the AI company didn't request, according to interviews with multiple people involved in the process and documents shared by OpenAI in response to questions from the Washington Post. A clone of actress Scarlett Johansson. This of course gets to the conversation that's been happening where Scarlett Johansson released a statement expressing concern that she had been asked by OpenAI to use her voice and when she said no, there ended up being a voice that kind of sounded like her. As I mentioned in a previous episode, I think there are multiple things going on here. There is the legal side of this, which at least this reporting from the Washington Post suggests that there might not be a there there. There's also just the broader question of the look and the public's trust, or lack thereof, in Sam Altman and Open AI. For now though, that is going to do it for today's headlines. Stay tuned for the main episode. Today's podcast is brought to you by Plum. Are you a lean product team trying to rapidly develop and deploy AI features that deliver real value to your users? Plum empowers you to build complex AI pipelines, transform data and leverage validated JSON schema to create reliable, high quality AI features accessible as API endpoints, all in an intuitive low code interface. Go from idea to MVP in hours, not days. Get your AI powered product in front of customers as soon as possible with plum. Check out useplum.com that's plum with a B for early access to the future of AI app development. Hello friends. Before we get back to the episode, I want to tell you about something special I'm doing on Super Intelligent this June. Super is of course our platform for AI learning and I've heard from a lot of you that you really want something for a true AI beginner, someone who's really just getting their feet wet with these tools. So what I'm going to do is put together basically a course that sits on top of and uses Super Intelligent tutorials and lessons, but where I hand guide you through around 10 different lessons and how tos that I think once you complete them will have you ahead of 80% of the other other people who are just starting to use AI right now. If you are interested in this learning experience, go to Be Super AI and sign up using code June. You'll get 25% off your first month and I'll automatically add you to that AI for Beginners group. That's Be Super AI discount code June. See you there. Welcome Back to the AI Daily Brief. One of the remarkable things about LLMs, this technology that has taken the world by storm, that is changing how people work, how people think about work, that is generating entirely new categories of interactions with computers, that has some people thinking that Terminator is going to become real, is that we genuinely don't understand exactly how they work. They just sort of seem to. Indeed, this is part of the reason for some researchers having concerns about the future state of these technologies. To wit, if we don't understand how they work now, how do we think we're going to control them as they get more powerful? While new research from Anthropic maybe shedding some light that will help us with that sort of understanding. The New York Times summed this up in a piece called AI's Black Boxes just got a little less mysterious. Kevin Roose writes, one of the weirder, more unnerving things about today's leading AI systems is that nobody, not even the people who build them, really know how the systems work. That's because LLMs are not programmed line by line by human engineers, as conventional computer programs are. Instead, these systems essentially learn on their own by ingesting vast amounts of data and identifying patterns and relationships in language, then using that knowledge to predict the next word in a sequence. Again, this is one of the great dividing lines in terms of how people think about AI risk. To some, this lack of understanding is precisely a cause for concern, while for others, perhaps most notably or at least Most loudly, Yann LeCun from Meta the current approach to LLMs that are just predicting the next word in a sequence are in his mind, simply incapable of the types of things that some folks are worried about. Holding aside any of the big long term existential risk things, there are challenges of our lack of understanding right now. The examples that the New York Times points out right now, if a user types which American city has the best food? And a chatbot responds Tokyo, there's no way of understanding why the model made that error or why the next person who asks may receive a different answer. So if you are a company building a chatbot trying to make it better, it's very hard to improve things in any sort of linear or controllable way. Of course, there is also the alignment side of this problem. As the New York Times Kevin Roose writes, When LLMs do misbehave or go off the rails, nobody can really explain why. From there, the Times talks about the field of research that is trying to figure out how these models work, which is called mechanistic interpretability. Roos characterizes the work as slow going, with progress being incremental. This week, however, Anthropic announced what they're calling a major breakthrough, and here's how Ruse sums it up. The researchers looked inside one of Anthropic's AI models, Claude 3 Sonnet, and used a technique known as dictionary learning to uncover patterns in how combinations of neurons, the mathematical units inside the AI model, were activated. When Claude was prompted to talk about certain topics, they identified roughly 10 million of these patterns, which they call features. This research actually started previously. Anthropic, in their announcement Post, writes, In October 2023, we reported success applying dictionary learning to a very small toy language model and found coherent features corresponding to concepts like uppercase text, DNA sequences, surnames and citations, nouns in mathematics, or function arguments in Python code. Now, however, they say we've successfully extracted millions of features from the middle layer of Claude 3 sonnet, providing a rough conceptual map of its internal states halfway through its computation. Whereas the features we found in the toy language model were rather superficial, the features we found in Sonnet have a depth, breadth, and abstraction reflecting Sonnet's advanced capabilities. We see features corresponding to a vast range of entities like cities, San Francisco, atomic elements like Lithium, scientific fields, immunology, and programming syntax. Like function calls, these features are multimodal and multilingual, responding to images of a given entity as well as its name or description in many languages. At this point in the piece, they show the Golden Gate Bridge feature, which activates around images of the Golden Gate Bridge or around text containing the Golden Gate Bridge. Anthropic goes on, we were able to measure a kind of quote unquote distance between features based on what neurons appeared in their activation patterns. This allowed us to look for features that are quote unquote close to each other. Looking near a Golden Gate Bridge feature, we found features for Alcatraz Island, Ghirardelli Square, the Golden State Warriors, California Governor Gavin Newsom, the 1906 earthquake, and the San Francisco set Alfred Hitchcock film Vertigo. They continue, this holds at a higher level of conceptual abstraction. Looking near a feature related to the concept of inner conflict, we find features related to relationship breakups, conflicting allegiances, logical inconsistencies, as well as the phrase catch 22. This shows that the internal organization of concepts in the AI model corresponds at least somewhat to our human notions of similarity. Importantly, says Anthropic, they're not just able to identify these features, but to manipulate them, artificially amplifying or suppressing them to see how Claude's response changes. Holding again with the example of the Golden Gate Bridge, they said, when initially asked, what is your physical form? Claude's usual kind of answer is I have no physical form. I am an AI model. But when amplifying the Golden Gate Bridge feature, Claude responded, I am the Golden Gate Bridge. My physical form is the iconic bridge itself. Altering the feature had made Claude effectively obsessed with the bridge, bringing it up in answer to almost any query, even in situations where it wasn't at all relevant, they continue. The fact that manipulating these features causes corresponding changes to behavior validates that they aren't just correlated with the presence of concepts in input text, but also causally shape the model's behavior. In other words, the features are likely to be a faithful part of how the model internally represents the world and how it uses these representations in its behavior, said Chris Olaw from Anthropic, who led this team. We're discovering features that may shed light on concerns about bias, safety risks, and autonomy. I'm feeling really excited that we might be able to turn these controversial questions that people argue about into things we can actually have more productive discourse on. An associate professor of computer science at mit, Jacob Andreas, who reviewed Anthropic's research, called it a hopeful sign that large scale interpretability might be possible, he said. In the same way that understanding basic things about how people work has helped us cure diseases, understanding how these models work will both let us recognize when things are about to go wrong and let us build better tools for controlling them. So obviously this doesn't tell us everything about how LLMs work, but it does give us a pretty strong jumping off point to go deeper in terms of this question of interpretability, sciency and dense though this may be, I think this is going to be an important part of how we resolve some of these questions of risk and challenges as AI moves forward. The longer we stay in the realm of theoretical debates, the harder it will be to actually put policies in place, whereas the more specific and applied we get, the better able we might be to actually solve some of the challenges. Super interesting stuff. Great work from the Anthropic team, but for now that is going to do it for the AI. Daily brief. Appreciate you listening or watching as always and until next time, peace.