Lemur and Lemurchat are new open-source models trained on code-heavy data that outperform other open models at blending natural language and code tasks. Baidu announced Ernie 4.0, claiming GPT-4-level performance across planning, reasoning, and multimodal generation.
Questions this video answers
A Llama-derived 70 billion parameter model (referred to as 'llama 270B' in the transcript).
2 trillion tokens of text data.
90 billion tokens of text and code with a 10:1 (text:code) ratio.
100,000 (100k) instances from both text and code data.
At Baidu World 2023 (announcement referenced in the transcript).
Chapters
Researchers from the University of Hong Kong, XLANG Lab, Salesforce Research, CAI Lab, University of Washington and MIT CSAEL have just unveiled something extraordinary. Lemur and Lemur Chat, two amazing AI models that can harmonize natural language and code for advanced language agents. Sounds pretty cool, right? Well, trust me, it's even cooler than you think. Also in the second part of the video, we'll discuss Baidu's latest AI, Ernie 4.0. They say it's as good as OpenAI's GPT4. But before we dive into that, let's talk about Lemur AI models. First of all, language agents are software programs that communicate using natural language to interact with humans or other agents. However, if you want to use language agents for more advanced tasks like searching for code snippets based on a description, generating code from a description or or translating documentation, it gets more challenging. These tasks need both natural language skills and coding expertise, which many language agents lack. Most current agents use large language models because they're good at understanding and generating natural language, but often struggle with code. Code has its own rules and is more exact than natural language, needing more logic and execution ability. So how can we bridge this gap between natural language and code? How can we create language agents that can handle both types of languages seamlessly? Well, that's exactly what this team of researchers have done. They have developed Lemur and lemurchat, two state of the art models that can harmonize natural language and code for advanced language agents. These models are based on llama 270B, which is a 70 billion parameter LLM that was trained on 2 trillion tokens of text data. However, the researchers improved this model by using a code centric corpus called the stacks, which contains 90 billion tokens of text and code data with a 10:1 ratio. This way they ensured that the model has enough exposure to both types of languages and can learn their similarities and differences. Lemur is the general purpose model that can handle various tasks involving text and code. Lemurchat is the specialized model that is optimized for dialogue use cases to create Lemurchat the researchers fine tuned lemur using 100k instances from both text and code data. They also used reinforcement learning with human feedback to align the model with human preferences for helpfulness and safety. Now, according to the paper, these models are pretty awesome. The researchers evaluated them on eight text and code benchmarks covering different scenarios such as code search, code summarization, code translation, text to code generation, documentation, translation, etc. They found that Lemur and Lemurchat outperformed all the other open source models on these benchmarks by a large margin. They also compared Lemurchat with some popular closed source models like ChatGPT and Palm on 13 agent benchmarks involving human communication, tool usage and interaction under different environments. They found that lemurchat significantly narrowed the gap with these models on agent abilities. This means we now have advanced models that can assist with tasks related to natural language and code. These models can help us develop smarter language tools for everyday use. They also let us explore how natural language and code work together. It's impressive that these models can communicate like humans and code like experts. I'm eager to see their potential in the future. Alright, now let's talk about a huge announcement that just came out of China. Baidu, the company behind China's largest Internet search engine, has unveiled its latest generative AI model, Ernie 4.0, and guess what? They claim that it's on par with OpenAI's GPT4, the most powerful AI model in the world right now. That's insane, right? Well, let me lay it all out and then you can judge. Alright, so Baidu has introduced Ernie 4.0, their latest AI model. They've been developing Ernie, short for Enhanced Representation through Knowledge Integration, since 2019. This model learns from different knowledge sources to better understand the world. At Baidu World 2023, CEO Robin Lee announced that Ernie 4.0 has greatly improved in understanding, generation, reasoning and memory. These improvements are crucial for AI applications and open doors for new innovations. Lee believes Ernie 4.0 matches GPT4 and meets human standards on many benchmarks. To demonstrate the new model capabilities, Lee showed some live examples of how new Ernie can handle different tasks and scenarios using its four core capabilities. He asked it to plan a family trip to Japan during cherry blossom season, and it quickly provided a detailed plan, including flight details and tips for enjoying the blossoms. He also gave it a text prompt about a dragon and a picture of mountains, and Ernie made an impressive artwork of a dragon flying over those mountains. When challenged with geometry problems, it not only solved them, but also explained its answers. Additionally, Lee had it write a martial arts story, and even as he kept adding new details, Ernie seamlessly included them in its narrative. The performance was quite remarkable, showing Ernie's versatility and ability to produce creative content. However, some experts were skeptical, noting it didn't seem much different from its predecessor, Ernie 3.0, and wondered how it stacks up against GPT 4. Baidu's CTO Wang Haifeng mentioned that since the model began beta testing in September, its performance has improved by almost 30%. It now has over 45 million users and has received good feedback. Baidu plans to incorporate generative AI across its offerings like Baidu Search, Baidu Drive and Baidu Maps. This will allow for more personalized user experiences. For instance, Baidu Search could give tailored answers instead of just a bunch of links. Baidu Drive can assist in organizing files using natural language commands. Moreover, new AI powered tools are on the horizon, such as the Baidu Wenku Smart Writer, which assists in content creation and and the Baidu Info Flow Smart Video Maker for easy video production. Baidu is embracing an AI focused strategy, aiming to transition from being purely Internet based to leveraging the power of AI. They see generative AI as pivotal in bringing innovative solutions to their users. But while generative AI offers many benefits and opportunities for innovation and creativity, it also poses some challenges and risks for security and regulation. That's why China has recently proposed some new rules and guidelines for managing generative AI services in the country. According to the Interim Measures for the Management of Generative Artificial Intelligence Services issued by China's Internet regulator in July this year, generative AI service providers must register their services with the authorities before launching them to the public. They must also conduct a security assessment of their services and ensure that they comply with the laws and regulations of China. China also has a blacklist of banned training sources, sources for AI models released by the Ministry of Industry and Information Technology. It includes sources with illegal or harmful info like violence, terrorism and more. The goal is to keep AI from generating harmful content and to ensure a healthy AI sector. In China. Along with this, China has guidelines like the Beijing AI Principles emphasizing respect for human rights, fairness, transparency and more in AI development for Ernie 4.0 and Baidu, this means they must follow these guidelines in China, ensuring their AI services are ethical and beneficial. However, it's also a chance for them to display their AI skills globally, offering value across different areas and competing with other big AI companies like OpenAI. So what are your thoughts on Baidu's generative AI strategy, especially Ernie 4.0? Can they compete with GPT4 and OpenAI? Share your thoughts in the comments. That's all for today's video. If you enjoyed if you enjoyed and learned something new, please like and subscribe for more AI content. And don't forget to hit the notification bell to stay updated. Thanks for watching and see you in the next one.
