Meta's Next-Gen AI Better than GPT-4 + Apple's NLP Breakthrough

So Meta has kicked off a huge new AI project, saying it's going to outshine GPT4 later on. We'll also chat about how Apple and Equal AI teamed up to boost the power of Transformer tech. Stick with us to the end to hear all about it, but first let's dig into what Meta's been up to. And hey, if you haven't subscribed yet, go ahead and hit that button to keep up with all the latest in AI. So what is this mysterious project that Meta is working on? Well, we don't have a lot of details yet, but we do know that it is a large multimodal model. Meta claims that this model will exhibit human level performance on various professional and academic benchmarks, such as passing a simulated bar exam or a Biology Olympiad, which is pretty impressive to be honest. But how is Meta able to achieve this level of performance? Well, one of the key factors is their massive investment in hardware and infrastructure. They have recently acquired a Large number of Nvidia H100AI training chips which are designed specifically for training large scale AI models. These chips are faster and more efficient than the ones used by OpenAI for GPT4. Meta has also built new data centers around the world to host these chips and provide enough computing power for their AI project. Another factor is Meta's transition from Microsoft's Azure cloud platform to their own infrastructure for AI training. This gives them more control and flexibility over their data and resources and allows them to scale up their AI project without relying on third party providers. Now you might be wondering, what about Llama 2? Isn't that Meta's latest AI model? Well, yes and no. Llama 2 is indeed a large language model that Meta released in July 2023 as an open source project. It is trained on 2 trillion tokens and has double the context length than the original Llama. However, Llama 2 is also a product of an uncommon alliance between Meta and Microsoft, while the new model will be entirely developed by Meta using their own infrastructure. So why did meta release llama2 if they are working on a better model? Well, there are several possible reasons. One is that Meta wanted to show their commitment to open source and cross collaboration in the AI community. Another is that Meta wanted to test the waters and gather feedback from developers and researchers who use Llama 2 for various purposes. And another is that Meta wanted to create some hype and anticipation for their new model by giving a taste of what they can do with AI. Now. Earlier this year, Meta put together a team to develop AI tools that imitate human expressions. This team is made up of experts in natural language processing, computer vision, speech recognition, and generative adversarial networks. They aim to design AI systems that can understand and produce natural language in various formats like text, images, audio and video. There's talk about this team creating an Instagram chatbot with 30 different personalities. This chatbot can interact with Instagram users on your profile, send product suggestions, gather potential customer details, offer discounts, and assist with customer questions. You can select its personality to match your brand, making it more relatable. All of this work ties into Meta's bigger plan to develop a Metaverse, a digital space where people can engage with one another and with online content in a lifelike manner. Meta sees AI as a crucial component for the Metaverse, providing the smarts and innovation to shape and adjust digital material. They think the Metaverse will be the next big thing in social media, shopping, entertainment and learning. Now let's talk about one of the most controversial aspects of Meta's new AI project, Open Sourcing. Meta has announced that they plan to make their new model open source and freely available for research and commercial use. This lets anyone use the model without having to pay Meta. This is a big step because many big AI models are usually kept private or have usage restrictions. Mark Zuckerberg, the head of Meta, claims that he wants to encourage more creativity and equal access in the AI world. He says he hopes that by sharing their model, more people can benefit from AI, and it will lead to more teamwork and healthy competition among AI creators. And he also thinks it will make the AI field more transparent and trustworthy, pushing for AI to be used in the right way. Now, while that's a noble sentiment, the genuine motivation behind this decision is likely that making the AI model open source allows experts from all over the world to contribute and enhance it. This not only boosts the company's image, but also sets the stage for a growing range of tools and services, and this can lead to new chances to make money. Plus, more people using the model can strengthen the company's reputation and lead to new partnerships. Meta's upcoming AI project is generating anticipation, and we're eager to know when it will be launched. And we might find out soon. On September 13, US Senate Majority Leader Chuck Schumer is holding an AI discussion. Key tech CEOs like Elon Musk, Sam Altman, Sundar Pichai, Jensen Huang, Eric Schmidt, Satya Nadella, and Mark Zuckerberg will be there. Schumer has scheduled nine of these talks to delve into major AI regulatory issues he'll be looking at how to safeguard workers, national security and copyrights, how to prevent worst case AI outcomes, and how to maintain fairness and responsibility in AI. Before making any AI laws. Schumer wants to gather insights from these experts. This event could be a perfect time for Meta to present their latest AI and its features. It's also a time for them to address potential critiques of their work. Moreover, they might use this as a chance to shape future USAI policies and regulations. Alright, now let's talk about Apple's and Equal AI's latest research streamlining the transformer architecture's feed forward network for better efficiency and accuracy. So you probably know by now that transformers are the hottest thing in NLP right now. They have taken over the field, especially in machine translation where they have achieved state of the art results on various benchmarks. Transformers are amazing because they can scale up really well. The more model parameters you add, the better performance you get across different NLP tasks. But there's a catch. Scaling up also means increasing the computational requirements, which can be a huge problem for real world applications. Think about it. You want your model to be fast, efficient and reliable, right? You don't want it to take forever to process your input or use up all your memory and disk space, or crash unexpectedly. That would be a nightmare. So how can we make these models more deployable and practical? Well, that's where the research comes in. A team of researchers from Apple and Equal AI decided to take a closer look at the transformer architecture and see if they could find any room for improvement. And guess what? They did. They focused on one of the main components of the transformer, the feedforward network, or FFN for short. The FFN is responsible for transforming each input token independently using some non linear operations. It adds complexity and expressiveness to the model's understanding of each word. Sounds important, right? It's a mixed bag really. The researchers discovered that the FFN actually exhibits a high level of redundancy, meaning that it does more work than it needs to. And that's not good because the FFN is also a large component of the model and consumes a significant number of parameters. In fact, it takes up about 2/3 of the parameter budget, leaving only one third for the other component. Attention. Attention lets the model understand the relationship between words in a sentence, no matter where they're placed. It's important for understanding language. So they chose to use just one FFN for all the encoder layers instead of having it in the decoder layers. This cut down on the number of parameters without losing accuracy. They also realized they could make the model smaller by adjusting the shared FFN's hidden dimension. By doing this, the FFN became wider but not as deep, and the model's performance and speed remained good or even got better. In simpler terms, this means we can make the FFN work better with less effort. We can also make transformer models easier to use and apply them to more language tasks without trouble. This research is truly a big deal. It highlights the possibilities in improving the transformer's design and reminds us there's still a lot to discover in language, language processing and AI. Alright, I hope you enjoyed this video, and if you did, please give it a thumbs up and share it with your friends. And if you have any questions or comments, feel free to leave them down below. Thank you so much for watching and I'll see you in the next one.