New AI Model Outperforms Google's Powerful PaLM-2

Inflection AI released Inflection 2, a multimodal model powering its PI chatbot that outperforms PaLM 2 on benchmarks. Trained on 5,000 Nvidia H100 GPUs for 6 months at ~$100M cost, it scores 81.2% on MMLU and handles sensitive topics more safely. PI has 10 million active users and 100 million conversations monthly.
Questions this video answers
2022
Reid Hoffman; Mustafa Suleiman; Karen Simonyan
Approximately 5,000 Nvidia H100 GPUs
About 10^25 FLOPs
FP8 mixed precision
Chapters
Inflection AI, the company that created the well known PI Chatbot, has recently upgraded the core model of its personal AI assistant, PI. Despite not receiving as much attention as it should, partly due to the buzz surrounding OpenAI and Sam Altman, this development is a major shift in the AI field. I'm excited to explain why this is so significant in this video, so let's dive right in. Alright, so Inflection AI is a company established in 2022 by prominent figures in the artificial intelligence industry. These include Reid Hoffman, who who helped start LinkedIn, Mustafa Suleiman, a founder of DeepMind and Karen Simonyan, DeepMind's chief scientist. Their goal is to develop a personal AI that can assist everyone. This AI can aid in learning new skills, solving problems and engaging in enjoyable conversations. Their initial offering is a chatbot named PI, accessible on their website or through a mobile app. PI operates using a sophisticated deep learning model named Inflection. This model has been trained using a vast range of information from different sources like books, Wikipedia, news articles, social media platforms and more. Now, Inflection AI has just announced the release of Inflection 2, the new and improved model behind PI Chatbot. Inflection 2 is a multimodal model, which means that it can accept both text and image inputs and output text responses. This feature makes PI more flexible and able to handle various tasks. For instance, PI PI can summarize an article, recognize objects in a photo, or write captions for images. This new model is significantly larger and more powerful than its predecessor and it was developed using 5000 Nvidia H100 GPUs. These GPUs used a technique called FP8 mixed precision and the total computing power used was about 10 to the 25th. Power flops. This level of computing power is similar to what Google used for its Palm 2 large model. However, Inflection 2 performs better than Google's model in most standard AI tests like MMLU, Trivia, QA, Heliswag and GSM8K. The design of Inflection 2 focuses on being efficient when put into use. Soon it will be the driving force behind PI. Inflection has upgraded from A100 to H100 GPUs and their methods for running the AI are very efficient. This upgrade means Inflection can do things cheaper and faster with inflection 2 compared to their previous version inflection 1. Even though inflection 2 is a lot bigger, this development is a significant step towards Inflection's goal of creating a personal AI for everyone. The company is really excited about what inflection 2 will bring to PI. As they progress, they are planning to build even bigger models using their 22,000 GPU cluster, so keep an eye out for what's coming next. Now, when it comes to building these huge models, Inflection has to be very careful about safety, security and and trust and they say they take these issues very seriously. Their safety team works hard to make sure their models are safe and follow the best practices. Inflection was one of the first to agree to the White House's voluntary commitments in July 2023. They're also involved in efforts to develop worldwide rules and systems to manage this important technology safely. Inflection 2 took about six months and cost about $100 million to train, which is significantly shorter and much cheaper than the time and money it took to train GPT4, which was about a year and around $1 billion. This new model has shown excellent performance across various tests, particularly when it hasn't been specially adjusted or trained for those specific tasks. It did really well on the MMLU benchmark, short for Massive Multitask Language Understanding. This benchmark is tough and new. It checks how well language models can handle 57 different tasks in areas like the humanities, science, technology, technology, engineering, mathematics and social sciences. These tasks range from solving math problems and answering trivia to writing essays. For a model to do well on this benchmark, it needs a broad understanding of the world and good problem solving skills. Inflection 2 got an average score of 81.2% on the MMLU, which is almost as good as the human score of 86.5% set by GPT4. This score is also much better than what Palm Large 2 and Llama 2 got, which were about 78% and 69% respectively. Now let's talk about how Inflection 2 improves PI and what it means for people using it and for the chatbot's overall performance. Inflection 2 makes PI better in many ways, especially when it comes to dealing with up to the minute information and delicate topics. The older version of PI struggled with information that wasn't available when it was trained, like recent news, trending topics, or new developments. Inflection two, however, can pull in and work with live information from different places like news articles, social media or Internet searches. This means PI can give more relevant and current answers to what you ask, and it also learns from what you tell it and what you're interested in. For instance, if you ask PI about the latest news, the weather or stock market updates, it will give you accurate and current responses. Another big improvement from inflection 2 is how PI handles sensitive subjects like race, gender, politics, and others. These topics are complex and can easily become contentious, so they need to be handled carefully to not upset or mislead people. Inflection 2 has been trained with more input from humans, including feedback from PI users, to better align with how people think and feel. It's also more aware of what it doesn't know or can't do, and it can choose not to answer or recommend talking to a human expert if it's faced with a question that's outside its abilities. For example, if you ask PI for its take on a moral dilemma, a legal matter, or a political argument, it will either respond thoughtfully and respectfully, or it will politely say it can't answer and suggest you look for a more trustworthy source. PI has become really well known and effective in the market, and this matters a lot for the AI industry. Since the release of Inflection 2, PI's popularity has soared. It has over 10 million active users and more than 100 million conversations each month. What's more, users keep coming back to PI. Over 80% of them return to chat within a week, and more than half recommend PI to their friends and family. Users have given Pie lots of positive feedback, praising its usefulness, friendliness and fun aspect. It's been highlighted in big media outlets like Forbes, Wired and the Verge as one of the most innovative and impressive chatbots out there. The launch of Inflection 2 also made a big difference in the AI industry. It showed how quickly and remarkably Inflection AI is advancing in deep learning and developing more complex and capable language models. Inflection AI is really focusing on AI safety and alignment, taking more advice from humans and experts and tackling the challenges and risks of large, powerful models. Inflection AI is now a big rival to OpenAI, the current frontrunner in AI, and is catching up in terms of model quality and capabilities. They're also working with other big companies and organizations like Nvidia, Microsoft and Facebook, using their resources and knowledge to create more benefits and impacts for users and society. Inflection AI and PI are planning some big things for the future. They actually said they want to create models that are 10 times bigger than their current inflection 2 model in the next year, which means models with trillions parameters. They also want to add new features to PI like audio and video, and explore areas like gaming, education and health. Plus they're working on making PI cheaper and more available to everyone. These developments will have a big impact on both the AI industry and society. We can look forward to new and exciting tools that can assist us in different ways and improve our lives. But it's also important to be cautious and responsible in how these models are developed and used. We need to make sure they match our values and don't cause any harm. We also have to be ready for any challenges they might bring, like ethical or legal issues, and work together to keep everyone safe. That's all for today's video. I hope you found it interesting and learned something new. If you liked it, please give it a thumbs up and subscribe for more AI and chatbot content. Let me know your thoughts about inflection, AI and PI in the comments, especially if you've used it or want to. Thanks for watching and I'll see you in the next one.
Sam.










