Google’s New AI Can Now Think at Superhuman Level (Scary Fast)

Google has several new AI developments that have attracted a lot of attention lately and one of the most significant focuses on something called mixture of Depths or mod, which addresses the problem of transformer based language models allocating the same amount of compute to every token in a sequence. The Researchers at Google DeepMind showed that it is possible to dynamically allocate blops by routing certain tokens through the expensive self attention and MLP computations, while allowing other tokens to skip them through a residual connection. The idea behind mixture of depths is actually pretty simple. Instead of treating every word in a sentence as equally important, this AI figures out which words need more attention and which don't. It then spends more processing power on the important ones and skips over the less important ones, saving time and making everything run faster. This means the AI gets just as good, sometimes even better, while using half the effort. It's like reading a book where you automatically focus on the important parts and skim over the filler without missing anything. The approach involves a per token router that produces a scalar weight for each token in a sequence. The top k tokens according to those weights get processed as usual by that transformer block's self attention and mlp, while the rest skip around that block in a less expensive route. This leads to a substantial reduction in flops since not every single token is treated equally. The entire procedure keeps a static computation graph because the capacity is preset to a specific K, ensuring that hardware utilization remains efficient. The tokens being routed to the block might differ from one sequence to the next depending on how relevant or difficult the token is. The MOD researchers set the capacity well below 100% of the tokens, for example 12.5%, which routes only 256 out of 2048 tokens to the full computations. They noticed that it is effective to do this only in every other layer in the transformer, making it possible to reduce the total compute overhead while still preserving or or even improving performance. The surprising outcome is that MOD matches or surpasses the baseline transformer in terms of language modeling loss and ends up being faster or bigger within the same budget because it uses fewer flops per forward pass and can allocate them to a larger parameter model or use them to train for more steps. The top K is determined by the router's output, and the researchers also discovered that this approach can reduce the memory footprint and lead to speed ups of 50% or more during post training sampling. One aspect they had to address was the question of how to perform autoregressive sampling when top K routing might usually require the entire sequence's router outputs. They introduced a small auxiliary predictor or a small auxiliary loss that approximates the top K decisions in a causal way, so that tokens can decide whether they route to or around that block without looking ahead. The performance cost is minor, on the order of 0.2 to 0.3%. The researchers also combined mixture of depths with mixture of experts to produce what they call mode, which can skip entire blocks for some tokens and also route them among different MLP experts. They found that this integration leads to improvements beyond simple capacity reduction in a normal MOE. They performed ISOfLOP comparisons at scales like 6e18, 2e19 and 1e20 flops. They saw that the MOD approach allows them to reach either the same or better training objectives than baselines, but at a fraction of the forward pass cost. In some configurations, the the model outperformed the baseline by up to 1.5% in the final log probability objective. They also noted that when the capacity is aggressively lowered, like 12.5% of the tokens being processed, there is often a gain in efficiency as long as there are still full capacity layers interleaved. The authors recognize that certain tokens simply do not require as much processing, so skipping computations on them frees the capacity for the tokens that truly need it. On a different front, Google has revealed that its VAO2AI video generation model will cost $0.50 for every second of video that is generated. This means $30 per minute, or around $1,800 per hour of AI generated footage. The service is described as a premium tool targeted at professionals and businesses and that need short yet sophisticated video content. John Baron, a researcher at Google DeepMind, offered a point of reference by comparing it with the cost of producing a high budget Hollywood movie like Endgame, which was reported to be around $32,000 per second, illustrating that 50 cents is relatively small compared to huge studio budgets. However, there is a possibility that users end up paying for generated seconds and that they never incorporate in their final videos, especially since VO2 can produce clips of about two minutes in length. The tool has been contrasted with OpenAI's Sora model included in a $200 monthly ChatGPT Pro subscription, which shows a different approach to billing. Another development on this same text to video technology is that FreePik began offering early public access to VO2 under its Creative Suite, where the first 10,000 users can try it with two free generations and then the cost is measured in credits each 5 second video requires 1000 credits and the cheaper Essential Freepik subscription is $69 per year which will get 84,000 credits corresponding to up to 84,000 credits corresponding to up To 84 short clips. This arrangement can be costly if a person needs large amounts of generated footage, but it is likely intended for short commercial style videos that can be used in presentations or marketing campaigns. In general, VO2 stands out for the realism of the content it produces, which is why Google charges a relatively higher rate. Another interesting new product from Google is the AI Co Scientist system, which is meant to help research scientists form new hypotheses, propose experiments, refine them and even produce thorough research overviews. It is built using a multi agent architecture on top of Gemini 2.0. It has agents called Generation Agent, Reflection Agent, Ranking Agent, Proximity Agent, Evolution Agent, Meta Review Agent and a supervisor agent that coordinates everything. One agent creates an initial set of hypotheses, another agent critiques them, another ranks them through a kind of tournament, and so on. They use an ELO rating process, the same system that often appears in chess, to rank these hypotheses. When a lower rated hypothesis beats a higher rated one, it gains more ELO points so the system keeps improving over time. The group behind this tool presented novelty and impact metrics that compare favorably to other AI approaches. The Co Scientist variant showed a novelty rating of 3.64 and an impact rating of 3.3.09, while the Gemini 2.0 Pro experimental setting sat at 3.27 novelty and 3.00 impact and OpenAI01 was noted to have 3.09 novelty and 3.09 impact. The co scientist version had the best combination and it scored a lower average ranking of 2.36, meaning it tended to be the top choice in pairwise comparisons. One of the main achievements that highlighted its potential was the way it solved a decade long mystery about antibiotic resistant bacteria in only two days. Professor Jose Panadis and his team at Imperial College London had spent about 10 years looking into how superbugs become resistant and they found that certain bacteria acquire viral tails that help them move between different hosts. When Panatis and his colleagues gave Google's AI Co scientists the same puzzle and it arrived at that exact theory independently in just 48 hours, even though it was not published anywhere. It also proposed four other plausible hypotheses which indicated that it was not just memorizing research. Penades initially believed that the AI might have had access to his private files, since the correct theory had never been publicly released. Google assured him that was not the case. The outcome convinced Penadez that this AI approach could change research permanently by ruling out dead end experiments more quickly and directing scientists to the most likely scenarios. There has also been a new report that Google is testing AI video creation features powered by Gemini. Tipsters found code references in the Google app about something called Robin, which apparently points to next level text to video generation. Observers discovered code snippets that talk about progress messages like working and and mention video generation specifically. Google has not confirmed how this will roll out to users, although there is speculation that a future Gemini version could let the user type a prompt and see a short clip. The current text to image component in Gemini uses Imagen 3, but the possibility remains that VO2 might show up in a more consumer friendly format later on. There would be an important question of cost, especially since these models can demand a lot of computing resources. Free or low cost versions might generate shorter and more limited videos, while a paid tier might unlock the full power. Access could merge with other Google offerings such as the business centric Google Vids or Google Workspace subscription tools like Google Vids for presentations or Marketing. The entire concept is reminiscent of the push to unify all sorts of AI functionalities under one roof, so Gemini might eventually handle 10 text, images and video with a single interface. Alright, so that's everything on Google's latest AI breakthroughs. What do you think about mixture of depths, VO2's pricing or AI co scientists solving a decade long mystery in just two days? Let me know in the comments. Don't forget to like and subscribe and I'll see you in the next one.