Microsoft’s New AI "MAI" Will Replace OpenAI Soon!

Microsoft spent billions backing OpenAI, but now they're building their own in house AI called Mai, possibly to dodge OpenAI's sky high costs with their new agents rumored to hit $20,000 a month. That's a steep price for advanced AI, and Microsoft's move suggests they're betting on their own tech to cut costs and take control. We'll break down Microsoft's Mai, those pricey OpenAI agents, and the fascinating belief state transformer that could change how AI plans and and thinks. So let's start with the brewing tension between Microsoft and OpenAI. Microsoft and OpenAI have been partners for a while, with Microsoft investing around 13 or $14 billion in OpenAI, helping them grow in the early stages. In fact, they were once so tight that Microsoft was OpenAI's exclusive cloud provider on Azure. But things are getting complicated now. Word on the street is that Microsoft has been training its own own advanced AI models that could rival OpenAI's GPT series, the ones powering ChatGPT and models from other big players like Anthropic. Various reports suggest Microsoft has developed a new family of large language models internally often referred to as mai, possibly short for Microsoft AI. They tested these MAI models to see if they could power Microsoft's Copilot system, a popular AI assistant that helps users write documents, manage spreadsheets, or even help in coding. Copilot currently relies heavily on OpenAI's models, but apparently Microsoft wants to reduce dependence on that partnership, possibly to cut down on costs and have more flexibility. Some sources say that MAI perform at least as well as OpenAI's and Anthropic's leading models on major benchmarks. And guess what? Microsoft isn't just testing its own LLMs. They're also trying out third party models from Meta, Xai, Deepseek, Anthropic, and more. Now it's not only about performance or bragging rights. One big factor here is money. Maintaining large language models, especially for products used by millions, means racking up huge data center bills. AI chips can cost anywhere from 10 grand to 30 grand a piece, and entire server racks can end up costing over half a million dollars before you even flip the power switch. This intense overhead has led to some eye popping pricing in the AI world. For example, OpenAI is rumored to be rolling out AI agents with monthly fees as high as $20,000. If you want a top tier PhD level research agent, that is absolutely wild. SoftBank, for instance, has apparently committed $3 billion to buy AI agents from OpenAI this year alone, believing it's worth it if these agents do the kind of scale skilled work that top paid experts do in finance and research. But not everyone is convinced that paying 10 or $20,000 a month is justified for these advanced AI assistants. Some smaller startups offer coding agents at around $500 a month, though these may not be quite as powerful. Still, that's a massive difference compared to the high end OpenAI agents. The question on everyone's mind is does the AI truly replace the need for human professionals or is it more hype? Meanwhile, Microsoft's pivot is also visible in how they changed their agreement with OpenAI. Earlier this year they moved away from being the exclusive Cloud provider for OpenAI, now simply having the right of first refusal. That means Microsoft can still host OpenAI's workloads if they want, but OpenAI is also free to use other cloud providers like Oracle. This is happening around the same time as big investments are going into AI infrastructure. Think the Stargate project, with Oracle, SoftBank and Nvidia possibly pumping up to $500 billion into next generation AI hardware. In the United States, Microsoft itself plans to invest around $80 billion in AI data centers in fiscal year 2025, including a healthy chunk going to India. All this background drama sets the stage for Microsoft's new internal push on advanced research, some of which comes down to a series of models called Phi and Mai, plus new developments in reasoning AI. The rumor is that these reasoning models tackle more complex tasks like multi step logical problems and could rival the advanced stuff from OpenAI or Anthropic. They may eventually be accessible through an API, just like OpenAI's GPT models, so other developers could integrate Microsoft's AI into their own apps. Now, if all that product and business competition wasn't enough, Microsoft Research has also been busy on a purely technical front. One of the biggest new ideas is something called the belief state transformer. This is an approach to sequence modeling, which basically means how an AI processes and generates a sequence of tokens or words with typical left to right models. Like most large language models we know, the AI sees tokens from the past but not from the future. The belief state transformer, however, uses both a forward encoder and a backward encoder. It can absorb information from the prefix, the words leading up to a certain point, as well as from a suffix, the words after that point. Then it can predict not only the next token but also the previous token in a sequence. Why is that such a big deal? Well, it turns out that tasks requiring planning, especially those where you need to know the final goal can really throw typical left to right models for a loop. One example is the so called star graph navigation problem. In a star graph you have multiple branches and you need to pick the correct path from a start node to a goal node. Forward own models often cheat by ignoring the goal node and guessing any valid next node. This yields mediocre results because the model never truly learns to plan from start to end. In theory, once a standard language model picks up this shortcut, it becomes as difficult to fix as solving complicated parity functions with gradient based training, which is basically super hard. The belief state transformer solves this by requiring the AI to also predict something about what comes before the suffix, so it can't cheat as easily during training. It sees a prefix and suffix and needs to produce both the next and previous tokens. That forced bi directional perspective helps it maintain a compact belief state that truly encapsulates all the info needed to generate correct C sequences. In experiments on star graphs, it drastically outperforms forward only baselines. Even other approaches like multi tech prediction or fill in the middle where you feed a chunk of text at the end to serve as a goal, can't keep up in challenging scenarios. The belief state transformer ends up learning an internal representation that's effectively a distribution of all relevant future states, letting it better handle tasks with inherent planning or goal oriented steps. Some small scale tests with a data set called Tiny Stories also showed that the belief state transformer outperforms typical fill in the middle strategies when generating coherent stories that match both a given prefix and suffix. The team behind this used GPT4 to compare sample stories, rating them on grammar, flow, coherence and creativity. BST based stories apparently had smoother transitions, ended properly, and maintained a more logical structure between the prefix and suffix. By contrast, many standard forward mode attempts ended abruptly or broke the story's logic. What's more, the belief state transformer can do unconditional text generation by scoring its own output from a previous token decoder, effectively letting it plan out a story step by step and check if it has a satisfying ending. This extra forward backward synergy can lead to stories that read more fluidly. Of course, all of this is still at a relatively small scale compared to large commercial models. The belief state Transformer has about 80 million parameters in some experiments, whereas GPT4 has hundreds of billions. But the principle could scale, and that's what's so exciting. If these ideas make it into large scale systems, it could give Microsoft a strong edge in building advanced AI that's better at planning, reasoning and generating top quality text. Between forging its own path with new LLMs like Mai, testing out external ones from Meta and Anthropic, and pushing fundamental research like the Belief State Transformer, Microsoft is doubling down on AI from multiple angles. They want Copilot to be powered by the best models, whether that means using OpenAI for certain tasks, using their own big reasoning models, or hooking in something else entirely to cut costs or improve performance. Meanwhile, that tension with OpenAI keeps ratcheting up. Microsoft is no longer shy about listing OpenAI as a competitor in areas like software, search and advertising, along with Google and various social platforms. It's a fascinating shift. Once upon a time they were more or less an exclusive power couple. Now they're more like frenemies, both collaborating and competing at the same time. And with OpenAI's new million dollar price tags for advanced agents, along with the high burn rate for GPU clusters, it's clear that both companies see gold in the AI frontier but also big risks. So where does that leave the rest of us? Well, we can probably expect more fierce competition, possibly more variety in big name AI services and maybe, fingers crossed, some downward pressure on those sky high monthly fees as more viable contenders emerge. Or it could be that leading AI providers stay expensive, focusing on enterprise customers who need the absolute best. Either way, it's going to be a wild ride anyway. Competition almost always spurs faster innovation and we might see some truly next level capabilities hitting products we use every day. Don't forget to like, share and subscribe. If you enjoyed this breakdown, thanks for watching and I'll catch you in the next one.