Google DeepMind's JEST AI Learns 13x Faster & SenseTime's New AI Beats GPT-4o

All right, super exciting stuff coming out of Google's DeepMind. So they've come up with a new way to train AI models that's not only faster, but also way more efficient. This method is called jest, which stands for joint example selection. And trust me, it's pretty revolutionary. So let's talk about it. Alright, so if you know anything about AI, you know that training these models usually takes a ton of time and a lot of computing power. It's like trying to teach a child every subject in school or one by one. And it's super energy intensive. For example, systems like ChatGPT guzzle a lot of power and water to keep those massive data centers cool. There are even comparisons to the energy demands of bitcoin mining. Yeah, it's that intense. But here's where Jess comes into play. The researchers at DeepMind have found a way to make this whole process way more efficient. How efficient? Like up to 13 times fewer iterations and 10 times less computation than that's a game changer for both speed and energy consumption. So how does JEST work? Instead of picking individual pieces of data to train on, which is what's usually done, Jest looks at groups or batches of data and picks the best ones that work well together. Think of trying to learn multiple languages. Instead of learning English, then German, then Norwegian separately, you might find it more effective to study them in a way where what you learn in one language helps you understand the others. That's essentially what JEST does, but with data. Here's a bit more on the technical side, but I'll keep it simple. The method JEST uses is called multimodal contrastive learning. This means it looks at different types of data, like images and text together, and it identifies dependencies between them. This approach significantly speeds up the learning process because the AI can learn from these well matched data batches much more efficiently. What makes JEST even cooler is that it starts starts with something called a pre trained reference model. This model helps steer the AI towards the high quality, well curated data, making the training even more efficient. Now, the study showed that using jest, they could achieve the same performance as traditional methods with up to 13 times fewer iterations. Also, it requires 10 times less computational power. So in terms of energy consumption, that's a massive reduction, which is great news for the environment. Now the just research comes not a moment too soon, as the tech industry and world governments are beginning discussions on artificial intelligence's extreme power demands. To give you an idea, AI workloads consumed about 4.3 gigawatts in 2023, which is almost equivalent to the annual power consumption of the entire nation of Cyprus. And this trend is only going up. A single ChatGPT request currently costs 10 times more in power than a Google search. It's estimated that AI might take up a quarter of the United States power grid by 2030. Therefore, any method like JEST that can drastically cut down on the power required for training AI models is incredibly significant, potentially easing the environmental impact and the financial costs associated with AI advancements. Furthermore, the success of the JEST method hinges on the quality of the training data. The bootstrapping technique utilized by JEST requires a highly curated initial dataset to function effectively. This means that while JEST shows great promise for large scale, well funded AI projects, it might pose challenges for hobbyists or smaller developers who lack the resources to compile high grade training data. Essentially, this method emphasizes the old adage of garbage in, garbage out, underlining the necessity for top tier data to achieve optimal results. If implemented widely, however, JEST could revolutionize how major players in the AI industry approach model training, but possibly reducing costs and power consumption while accelerating development timelines. Another neat thing they found is something they call data quality bootstrapping. It sounds fancy, but it's pretty straightforward. It means using a small, highly curated set of data to guide the training on a much larger, messier set of data. So even if you start with a tiny, really good data set, Jess can help you scale up to train on a vast amount of data efficiently without losing quality. This is a big deal because high quality data sets are usually hard and expensive to create. Now, this method isn't just limited to one type of data or task. It works across various benchmarks, which means it's versatile. For example, they tested it on a data set called WebLI, which is a large scale collection of images and text from the web. And the results showed remarkable improvements in both learning speed and resource efficiency. So if JEST proves effective on a larger scale, it could drastically reduce the power needed to train AI models. This means we could develop more powerful AI tools with the same resources we use now, or use fewer resources to create new models. Essentially, we're looking at faster advancements in technology, less strain on our energy resources and a smaller environmental footprint. It's a win, win, win. Alright. Now you know the big AI companies in China have been making some major claims about their large language models at this huge AI conference happened recently. It's like a competition to see who can boast the most impressive advances. You know what I mean, so there's this company called SenseTime. They're kind of big shots in the AI game over there. They just unveiled the latest versions of their sense Nova LLMs. And get this, they're claiming the new 5.5 model is like 30% better than the previous one from just a few months ago. Wild, right? The CEO, Shu Li, he was on stage hyping it up big time, saying the key to success in Chinese AI is basically building these high level logic systems to using synthetic data for different industries. Xu claimed that this 5.5 model supposedly outperformed GPT4O, the latest and greatest from OpenAI, in five out of eight key metrics, according to some benchmarking data from a platform called Open Compass. I mean, take that with a grain of salt for sure, but still, that's a pretty big flex. Flexing on OpenAI like that is a gutsy move. Now, despite making these huge claims about their model's performance, SenseTime's stock price actually tanked like 16% on the same day in Hong Kong. Crazy reversal, right? You'd think investors would be stoked about supposedly having a model that can take down GPT4, but I guess the market had other ideas. And it's not just SenseTime making noise at this conference. Oh no. Alibaba's cloud division, the folks behind the Tongyi Chanwen LLMs, they were bragging about getting like millions of new downloads and a surge in customers for their AI platform in just the last couple of months. The cto, Joe Jingren, he was really pushing this whole open source angle hard, saying their models are now fully open source and closing the gap with closed source ones from the big Western players. Smart move, if you ask me. Makes it more accessible and gets more eyes on their tech. But here's the wildest part of it all. There's this AI startup CEO Yan Junji, from a company called Minimax, and he straight up predicted that in the future there will only be like five companies making these huge LLMs globally. Now that's a hot take if I've ever heard one, right? I mean, could you imagine that kind of consolidation happening? Just five players controlling the entire large language model space worldwide? Insane to think about. Of course, Yan didn't actually say how many of those five would be Chinese companies, so we'll have to wait and see on that one. But the implication is clear. He thinks there's a real chance for Chinese firms to be among that tiny elite group someday. Whether that's realistic or not, I'll leave that up to you to decide. Oh, and back to SenseTime for a sec. They also showed off a couple of other new models at this big Shanghai conference. The first is called 5O and it's this multimodal model that can combine different data types like text, images, video, all that jazz into one seamless experience. And they did a live demo with the 5o that was low key, mind blowing, not gonna lie. So there's this SenseTime employee on stage, right, and he takes a video of himself just standing there, seemingly nothing special. But then the 5O model instantly identifies that he's attending this World AI Conference thing just from analyzing his staff badge and T shirt in the video. It's wild, but it gets even crazier. The model could then answer follow up questions about what the conference is for and all the deets like it achieved true understanding from just that little video clip. I'm talking next level multimodal AI capabilities on display here folks. So yeah, between the bold performance claims, the open source pushes, the predictions of an LLM oligopoly, and the awesome tech demos, it's been an absolute whirlwind of big moves and even bigger ambitions from the Chinese AI giants at this conference. And that's not even getting into what companies like Baidu and Tencent were unveiling. It's definitely an exciting time in the AI space, that's for sure, and it'll be fascinating to see how all these moonshot plays actually pan out in the real world. Can the Chinese models really take down the Western juggernauts? Will we see a small group of ultra dominant LLM providers emerge globally? So many intriguing possibilities. One thing's for certain though, these Chinese tech companies are clearly not messing around. They've got their sights set on AI supremacy and they're going all in to make it happen. Whether that's through cutting edge R and D savvy business strategies or sheer bravado and ambition. Alright, if you found this interesting and want to stay updated on more AI insights like this, don't forget to subscribe to the channel. Thanks for watching and I'll see you in the next one.