Llama 3 Is Here (And Seemingly Better Than Expected)

Today on the AI breakdown, the anticipated Llama 3 release has just happened. Before that, on the brief, Microsoft's Vasa 1 model is turning heads. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our YouTube, our Discord and our newsletter. Welcome back to the AI Breakdown Brief. All the AI headline news you need in around five minutes Yesterday everyone started talking about a new model from Microsoft research called Vasa1. Binduretti summed it up this way. She called it the first AI generated video that looks super real and said it takes a single portrait photo and speech audio and produces a hyper realistic talking face video with precise lip sync audio, lifelike facial behavior and naturalistic head movements generated in real time. This is amazing given that the AI generated video looks very real. So for those of you who are watching this, rather than listening to it, let's check out a quick example you ever had.

Maybe you're in that place right now where you want to turn your life around and you know somewhere deep in your soul there could be some decision.

It really is incredibly lifelike. And indeed that's the way that they describe it in their research paper. The paper is called Vasa one Lifelike Audio Driven Talking Faces Generated in Real Time tldr, they write, single portrait photo plus speech audio equals hyper realistic talking face video with precise lip sync audio, lifelike facial behavior, and naturalistic head movements generated in real time. In the abstract, they write, the model is capable of not only producing lip movements that are exquisitely synchronized with the audio, but also capturing a large spectrum of facial nuances and natural head motions that contribute to the perception of authenticity and liveness. The core innovations, they say, include a holistic facial dynamics and head movement generation model that works in a face latent space and the development of such an expressive and disentangled face latent space using videos. Part of what they're so excited about is that the model not only delivers realistic lifelike videos, but supports, as they write, 512x512 videos at up to 40 frames per second with negligible starting latency. What that means is that conceivably we're not too far away not only from lifelike AI generated videos, but from actually being able to interact with these lifelike avatars in real time. Now, in addition to producing just lifelike videos from real photos, they say that their method also has the ability to handle photo and audio inputs that are out of their training distribution. For example, they can handle illustrations or paintings, singing, audio or non English speech, none of which were present in the training set. Again, for those of you watching, here's an example of that lack of latency which makes it opportune for real time.

But you know what I decided to do? I decided to focus on all my attention, all my time on listening. So instead of doing something else, I just listened, listened and listened. Because I'm a true believer that if you're really bad at something now the.

Paper points out that this is not a technology without risk. The closer we get to truly lifelike and especially real time generation like this, the greater the chance that it's misused for disinformation or misinformation for impersonating people without their permission. And at this time, they write, we have no plans to release an online demo API product, additional implementation details, or any related offerings until we are certain that the technology will be used responsibly and in accordance with proper regulations. And ultimately the biggest caveat or asterisk on any of this is that none of us have actually gotten a chance to play with it. This is just a paper with cherry picked examples that, while super impressive theoretically might not represent the standard output that you would get if everyone was allowed to use this. Still, it does seem like a fairly significant jump in this sort of avatar capacity, and so is very much something to be watching for in the future. Next up we get an update from OpenAI. The company has released a new version of its Assistance API. Now the Assistance API is OpenAI's tool that allows people to build agent like assistants that have some specific purpose that they use ChatGPT for. One of the big changes is as Brian Romley writes, the new OpenAI Assistance API can access up to 10,000 documents in a vector database rag. It's quite useful and I have already mentioned it to clients that need larger scale rags. Saliomar from Cognosis says new OpenAI assistant updates are really good. Hard to bet against them. Tested retrieval and it's insanely fast and almost instant. Seems like they want a piece of the RAG pie as well. At what point does it become cheaper to use them versus building your own? For those of you drowning in acronyms, RAG stands for Retrieval Augmented Generation and basically refers to the idea of an LLM pulling from a specific set of information for some specific purposes. This is a popular strategy right now for for example, enterprises that want some version of an LLM to be able to pull from their proprietary information in order to give people in the company insight that is specific to the company. There are tons and tons of solutions for how an enterprise might go about that, and Sully's pointing out that the better that OpenAI and ChatGPT get at it, the more incentive that companies have just to stay in that ecosystem. Of course, ultimately it's not just a question of technical capacity, but a question of data trust. And my perception is at least that the biggest reason that enterprises are choosing not OpenAI to spin up their own models to customize open source models is because of those concerns around privacy. Still, like Sully says, the fact that this is an easy and fast approach that works really well could shift the balance of that conversation a little bit. Lastly today, an interesting one from the world of entertainment. There's been a lot of discourse in the entertainment sector and in Hollywood about artificial intelligence. Last year, the writer strike and the SAG strike were some of the first instances in which concerns around AI really started to get into the mainstream. My question at the time was not so much if the concerns were real or not, but whether if the choice was on the one hand try to ban or prohibit all this technology or on the other try to profit from it, how long would it take before Hollywood shifted over to the try to profit from it side? Apparently the creative artist agency caa, one of the big Hollywood talent agencies, is thinking in a similar way and testing a new program called CAA Vault that allows the talent they represent, or at least a small handful of A listers that are testing it to create a digital double of themselves that they can profit from, said Alexandra Shannon, CAA's head of corporate strategic development. On one hand, there's concerns that technology is being misused to exploit name, image, likeness, voice, body of work without consent. But we also recognize that it's creating opportunities for talent and an explosion of creativity in so many different ways. Shannon said. These technologies should not devalue the human. If somebody's digital likeness is being used in a campaign instead of them in person, it is still the value of that person and what they stand for as a representative of a brand. Of course, the concern that many have is that if the marginal cost of production comes down and there is a much larger supply of celebrity spokespeople in the form of these digital doubles, what will that actually do to the value of any individual instance of that? It seems like a pretty price deflationary force. Of course the Brad Pitt brand, for example, is going to continue to retain value. But if there are digital Brad pits running around everywhere, how much each individual instance of that is going to be able to charge. Ultimately, those are questions for caa, not for us, and so that is going to do it for the AI Breakdown brief. Next up, the main AI Breakdown Today's podcast is brought to you by Plum. Is your product team struggling to keep up with the incredible pace of AI development? Are you tired of spending countless engineering hours just to test out small prompt changes in your product? Thankfully, there's Plum. Build cutting edge AI experiences for your users in a fraction of the time. Say goodbye to the slow, tedious process of hand coding and hello to the future of AI development. Get ahead of your competition and start moving as fast as AI does. Check out useplum.com and shoot me a message to get early access Attention AI breakdown listeners. ConsenSys 2024 marks the 10th gathering for all things crypto, blockchain and web3. However importantly, this year's agenda will also dive deep into AI driven transformation and the speaker lineup includes the leading minds and innovators at the forefront of this digital renaissance. Don't miss the Consensus AI Summit to cut through the hype to find where true transformation and opportunity lie. Listeners to this show can get 15% off registration with the code AIBREAKDOWN. Visit consensus202Fouresk.com to learn more. Some of the folks we'll be at Consensys this year include Guillaume Verdon, AKA Beth Jesos, founder and CEO of Xtropic, as well as spiritual leader of the accelerationist movement Neal Stephenson, co founder of Lamina One, and Brendan Ike, the CEO of Brave Software. Again, go to consensus2024.coindesk.com to learn more and get 15% off registration with the code AI BREAKDOWN. Welcome back to the AI Breakdown. Last week reports came out that Meta was on the verge of releasing its latest open Source LLM Models, llama3. Now, at first this was an unconfirmed rumor, but then within a couple days Meta seemed to indicate that yes, this was coming, or at least a set of small versions of their next model were coming in advance of the largest version, which would be coming out over the summer. All week then, we have been waiting with bated breath to see what Meta would put out, with tons and tons of speculation on just how good it would be. A couple hours before I recorded this episode, we started to get hints that Llama 3 was about to be dropped. First we saw Llama 38B instruct be listed on the Azure Marketplace with little nuggets of information like this line. The fine tuned versions are optimized for dialogue use cases. And then on replicate.com, we saw pricing for four different models. Llama 370B, llama 3.8B, llama 370B chat and llama 3.8B chat. People got in their last polls on how good others anticipated this to be. Jan Pelig wrote final moments to guess llama 3 will be and then gave the options State of the art Open Source software same level as state of the art Open source software, same level as state of the art Closed software or Outperforming State of the art Closed software Outperforming State of the art closed was the lowest ranked option with 8.8%, same level as state of the art closed had 12.8% and then state of the art open Source and same level as state of the art Open Source were almost exactly the same, getting 38.9% and 39.6% of the vote respectively. Accelerate Harder asked a simpler version of the same question. Llama 3 will be either amazing or a letdown, with 47.2% saying a letdown and 52.8% saying it would be amazing. Just a few minutes later we got the actual announcement. The AIHETA account on x wrote introducing Metalama 3, the most capable openly available LLM to date. Today we're releasing 8B and 70B models that deliver on new capabilities such as improved reasoning and a set of new state of the art for models of their size. Today's release includes the first two Llama 3 models. In the coming months we expect to introduce new capabilities, longer context windows, additional model sizes and enhanced performance. Plus Llama 3 research paper for the community to learn from our work, Chief AI Scientist at Meta, Jan Lecun gave a few more bits of information. He said 8b and 70b models available today, 8k context length trained with 15 trillion tokens on a custom built 24k GPU cluster. Great performance on various benchmarks with llama 38b doing better than llama 270b in some cases. We'll come back to that 8k context length because it sticks out kind of like a sore thumb relative to other models we've had recently. But as he has started to do, Zuckerberg took to their own networks to talk about the new release. He wrote on Instagram. Big AI news Today we're releasing the new version of Meta AI, our assistant that you can ask any question across our apps and glasses. Our goal is to build the world's leading AI. We're upgrading Meta AI with our new state of the art Llama 3AI model which we're open sourcing with this new model, we believe Meta AI is now the most intelligent AI assistant that you can freely use. We're making Meta AI easier to use by integrating it into the search boxes at the top of WhatsApp, Instagram, Facebook and Messenger. We also built a website Meta AI for you to use on web. We also built some unique creation features like the ability to animate photos. Meta AI now generates high quality images so fast that it creates and updates them in real time as you're typing. It'll also generate a playback video of your creation process. Enjoy Meta AI and you can follow our new Meta AI Instagram for more updates. So a couple things that are notable from this one Meta is continuing to reinforce their message of open source and that's something we'll see even more in some other parts of the announcement in just a minute. Second as had been intimated, Zuckerberg is clearly not content with just being the state of the art for open source. They want to go after the state of the art in general. You can see the sort of big language they're using. Our goal is to build the world's leading AI with this new model. We believe Meta AI is now the most intelligent AI assistant that you can freely use, even if there are little caveats in there, that is you can freely use. The ambition to be the best full stop is clearly on display. Another really interesting thing, however, from this announcement is the extent to which they are integrating this into products right out of the gate. This is clearly not meant to just be a developer release, but something that is immediately impacting consumer products. As of today, alongside the announcement, they put out a lot of new information. Some of it's for developers, but some of it of course is benchmarks. Almost immediately people's eyes started bugging out at those benchmarks. Matt Schumer writes, Holy S Llama 370B cleanly beats Claude 3 sonnet small enough to host at scale without breaking the bank. What he's referring to is both The MMLU where Metalama 3.70B claims an 82 versus Claude 3 sonnet 79 and human eval where Metalama 3 claims an 81.7 versus Claude 3 sonnet 73. Bindu ready from Abacus writes, historic moment. Llama 370B numbers are insane at 82 MMLU, it's far and away the best open source model. GSM8k math and human eval are mind blowing as well. The open source community is definitely going to beat GPT4 in a matter of weeks. Schumer also pointed out that Llama 370B cleanly beats Mistral 8x22B. We've talked a lot on this show about the extent to which Mistral had stolen some of the open source thunder from Meta over the course of the end of last year, and there certainly seems to be Meta's clapback overall, Professor Ethan Mollick writes, Meta released their open source AI Llama 3 today as a key leader in LLMs. Their models are often the most advanced open source ones out there based on benchmarks. The current model is not quite GPT4 class, but their larger ones still training will reach GPT4 level and indeed this is what some people are the most excited about. Schumer again writes the craziest Llama 3 reveal the 400B version of the model is on par with Claude 3 Opus and it's still training. Soon we'll have a better than opus fully open source model. The Implications are huge. Meta 3 400B's reported MMLU score is right on par with Opus 3 with their grade school math score and human eval score right around there as well, Aston Zhang from the meta team wrote. Llama 3 has been my focus since joining the llama team last summer. Together we've been tackling challenges across pre training and human data, pre training, scaling, long context, post training and evaluations. It's been a rigorous yet thrilling journey, he continues. Scaling is the recipe demanding more than better scaling laws and infrastructure. For example, managing high effective training time across 16k GPUs requires innovative strategies. Still, I think the big implications are summed up by Dr. Jim Fan from Nvidia, who writes the upcoming Llama 3 400B will mark the watershed moment that the community gains open weight access to a GPT4 class model. It will change the calculus for many research efforts in grassroots startups. Llama 3 400B is still training and will hopefully get even better in the next few months. There is so much research potential that can be unlocked with such a powerful backbone. Expect a surge in builder energy across the ecosystem. Now I think that's absolutely true. In many ways the story of 2024 so far has been this standardization of the GPT4 class models and so it's perhaps not surprising that open source seems to be catching up there as well. However, surprising or not, as Jim points out, the implications in terms of what people can build and how and for how much are fairly significant. Now, I mentioned that to the extent that there has been any quick Critique, it's that 8k context window. Some pointed out that they thought that the community would extend that pretty quickly and others inside Meta explained it as well. Aston Zhang again wrote, we've set the pre training context window to 8K tokens. A comprehensive approach to data modeling, parallelism, inference and evaluations would be interesting. More updates on longer contexts later. In other words, there are clearly trade offs that they were willing to make based on their goals with this release. Now speaking of this release, one other really interesting little detail Meta seems to have taken it to heart to go directly to the community because in addition to all the traditional PR strategies, Zuckerberg popped up on a number of creator shows that are beloved and well known inside AI circles, but not even close to the size of mainstream outlets. Roberto Nixon, who runs some of the most popular Instagram and TikTok channels on AI and future technology, did an interview with Zuck and Dwarkesh, whose podcast has quickly become, I think, the most high value interview show that exists, released an hour and 20 minute episode with Zuckerberg that gets deep not only on llama 3, but as Dwarkesh writes, open sourcing towards AGI, custom silicon, synthetic data and energy constraints on scaling along with, you know, intelligence explosions, bioweapons, $10 billion models, and much more. Overall, I would say the first impressions are extremely exciting. I think even more exciting than the community thought they were going to be when they heard that a couple of small models were coming last week. I'm sure in the coming days I will be able to add more context around what people are finding in terms of actual performance, but for now it's a cool day with lots to explore. That is going to do it for today's AI breakdown. Until next time, peace.