Ep. 038 - Is GLM 5.3 a Mythos-Class Cyber Model From China? (InferenceX)

Hello everyone. Welcome back to Semi-Analysis Weekly. This week I'm joined by Wenyao and Kimbo. We're going to talk about everything GLM 5.3. Guys put out a great article recently. The model is about 2 months old and we've got everything from InferenceX performance results to model architecture, post-training pipeline, and just like what people are using it for that we can dig into. Spoiler alert, lots of discussions around cybersecurity capabilities with this model. Kimbo, Wenya, welcome to the show, guys.

Thanks for having me, Jordan.

Excited to be here.

I can hear it in your voice, guys. Okay. So the title of the article was How GLM 5.3 Sparse Attention Affects HBM and Memory Usage. For those who don't know, maybe we can start, Kimbo, with you with just an introduction, like What is sparse attention? Why do we care about HBM memory usage when we talk about these new open models, especially ones that are coming from Chinese companies like Zed AI? Yeah.

Yeah. So sparse attention is a category of attention mechanism where in contrast to linear attention, where you try to modify the core operation of the attention, which is usually called a scaled dot product attention. Linear attention changes— tries to change the fundamental algorithm to linear time algorithm, whereas sparse attention chooses to change the way you select the number of tokens while keeping the core attention operation. So the idea is to invent some sort of select— token selection algorithm so that regardless of how long the sequence of context becomes, you would be attending to only the most important tokens. the most relevant tokens at every, at every forward pass. Awesome.

Yeah. So this makes the model a lot more efficient, right? And a lot of different labs are actually pursuing things like this to make attention a little bit more efficient. Can you maybe give a little comparison of what Zed AI is doing with the GLM models to some other labs like DeepSeek or KIMI?

Yeah. So like you said, it's almost in some sense inevitable because just physics, we just don't have that enough memory in that. And don't have that enough compute. And GLM uses, since GLM 5, they use the DeepSeek sparse attention, the attention mechanism DeepSeek invented actually for their V3.2 model. And they opted that instead. And the core components are, you have a Lightning indexer, which is the token selection component, and then the sparse attention. The sparse attention is based off of the more standard multi-latent attention, multi-head latent attention invented by DeepSeek at a— popularized on DeepSeek V3. So that's GLM. And for DeepSeek, even though they invented DSA, DeepSeek Sparse Attention on V3.2, they continued innovating. They continued changing their attention variants. So for V4, for the V4 series, they invented CSA, compressed sparse attention, and heavily compressed attention, which are most, roughly speaking, building on top of DSA and further adding more token compression modules. And for V4.1 Flash, they simplified the HSA and CSA. into like a slightly different design. And yes.

Yeah, sorry. I mean, I don't mean to interrupt you, but let's say that the, the reason they're pursuing this is in the name of efficiency, right? Um, like the, I think people speculate as to why the Chinese labs may care about this more than the American labs due to restrictions that they have on hardware, but roughly the outcome from all of these pursuits of sparse attention is that the cost to serve tokens on this chart, a million tokens goes way, way down. Like it's really, really cheap to serve if you have the ability to use sparse attention or like additional ways to like HSA and stuff like that, right?

Yeah, totally. It's just infeasible to fit that many tokens with a dense uh, like a dense attention, uh, anymore, given like our usage goes easily above a million, uh, token context for like a single session. Um, so, so yeah.

Okay, Wenya, let me bring you in here. When it comes to actually serving this model and trying to analyze the cost to serve, um, a million tokens at 150 tokens per second, this obviously involves us running benchmarks that simulate A workload that, you know, a real provider like ZenAI or somebody else who's downloading the model and serving it, like one of the inference endpoint providers, would actually be, you know, incurring. In other words, they want to serve dozens or hundreds or even thousands of concurrent users on a single model endpoint. Can you talk to me about that restriction that Kimbo just mentioned? Like, if you're going to support all these users with a million token context, it's going to have have a lot of pressure on your system, like the resources that you have available, like memory, right?

Yeah, yeah, absolutely. So I think for serving, one of the factors that people care the most is, like, one of the factors is definitely aggregated token output because it represents how many total output it can produce for your users, right? And there's actually a trade-off here is that Well, we only have this fixed amount of memory and the GPU. And in order to get more concurrency to serve more users, we need more memory. So there's a technique called high cache, which it serves as prefix cache reuse. And when there are more concurrency, the concurrency goes up. It will offload the memory from HBM to DRAM, and that ends up, will make the percentage of host memory goes up as the number of concurrency goes up. So if you see on the B200 charts that we have on the high cache, yeah, right here, the concurrency of 8 and 16 was very different on the percentage of host memory. And this is actually even larger than B300 because B200, it has less memory per chip. And yeah, it's a really interesting and, uh, observation from our side.

Kimbo, do you remember when high sparse and high cache first got introduced? Like these are Clearly optimizations that are meant to save people money when they're serving these models, which kind of implies that the people that are introducing them have a lot of users that are using their models that they need to worry about. It's not a theoretical thing.

Yeah. So HiSparse is the sparse attention version of HiCache. And the whole idea of HiCache is trying to offload KVCache from the HBM to a lower tier, higher capacity, but a lower bandwidth memory and only load the most relevant set of tokens during computation. And I think HiSPARSE is a good example of an explanation to, I think, also a misnomer that sparse attention fixes all the problems of heavy cache consuming too much memory or having high memory requirements. So concretely, sparse attention doesn't mitigate the memory capacity bottleneck. It's a technique that helps a lot with memory bandwidth, but as for capacity, because during the sparse attention computation, you would still need to select the tokens, so you still have to keep all the tokens in the relevant context and you cannot discard them during computation. So at some point it still has to be loaded into HBM, so the memory capacity consumption will not be significantly reduced. And that's where HiSPARSE becomes useful because it can apply the typical caching rules so that you could load the most relevant parts into HBM.

Okay. Wenyao, talk to me about some of the initial performance results here. I mean, like the ability to use HiSPARSE versus HiCache. I mean, both, I guess, depending on the definition of terms on this model, GLM 5.3. Is obviously giving some great performance results, especially for particular vendors. When you just look like total tokens per $1 of TCO, 45 million tokens at peak up here, like very, very efficient.

It is. It is. And I just want to add up to what Kimbal just said. I think one thing that people, that we want to emphasize on the article is that even though GLM is compute efficient, it doesn't mean that we have efficiency on the memory side. And because the model still needs to read through the relevant context history, right? So hotspots becomes very, very important. And it's offloading from HBM to DRAM and offloading back is very techniques And we've also introduced in the article. And as for the performance we have, I think generally NVIDIA chips just get ahead of all the other chips at the very beginning. And a few weeks later, when we about to ship the article, very surprisingly that the AMD interactivity at 150 tokens per second is relatively the same cost as NVIDIA chips on V200.

Yeah, let me, let me actually show that animation as we go through it right now. Like the model was released in early October and we've got this nice feature on InferenceX where you can go back in time and see how things make progress. So all the way through, sorry, it was released in early August. We're now in October. As you kind of shift through and you start to see these results show up on the dashboard, You know, wow. You can see the red AMD curves overtake the green Nvidia curves, which is quite impressive. And now Nvidia start to come back up, but just explain to me your experience here, 'cause you guys are running this stuff every day. Lots of people have been anti-AMD for a long time, especially the AMD software ecosystem, including us. But this is a relevant model that's not even 2 months old. And the best curves for the highest, you know, throughput, even at 100 tokens per second per user, which is a real interactivity range that, that, you know, the inference endpoints I can see on OpenRouter are not approaching. So it's below that. That's a lot of high performance from the MI355 from AMD. Good for them.

Yeah, they're definitely working hard overnight and trying to do a range of optimizations. I think it shows that, I guess there are 2 sides, right? The first side is the hardware where hardware vendors like NVM, they have their chips ready and For their customers, that chip's performance was fixed. And on the other side is the inference engine side where you have to be caught up and continuing doing optimization and iterate it over again to making sure that your performance stays more cost efficient. And this is a perfect example where we see that AMD engineers working hard on their engine Atom and over time. Its cost per million tokens just catch up to a lot of the NVIDIA chips.

Yeah. And we have a live chart specifically looking at the prefix cache reuse that you guys were talking about earlier. If you click this at the bottom, you can see for a given configuration like MI355 on FP4, flip over to GB300 for the given points on the curve, how much of the host cache, like the DRAM, is actually being used. Maybe dig into the point you were just making about the implications for DRAM. Like if we go back to the first big open model release from a Chinese lab, which was like DeepSeek taking the world by storm almost 2 years ago now, right? But anyway, V3. Yeah. And R1 memory stocks sold off in NVIDIA and everybody was really concerned that a more efficient model would mean less use of memory. But maybe the opposite is true where when you guys are describing, as you make the models more efficient, you can therefore increase concurrency and serve more users and you have just as many demand, as much demand on the memory itself. But could you put a little bit of a finer point on the capacity versus bandwidth thing? Because a lot of our listeners maybe tuned in for the conversation with Myron a few weeks ago where we talked about the despec of the Rubin GPUs for Rubin Ultra from a terabyte of memory per GPU down to 192, matching the R200, that keeps the bandwidth the same, but reduces the capacity. Can you explain the trade-off in the context of like inference where HBM bandwidth and capacity versus host DRAM bandwidth and capacity are all used to serve users at a lot of points on this curve?

I guess I can first start off and, you know, on the inference side, so if we think about Memory offloading and capacity. We definitely want to serve more users with more concurrency, right? Because that makes the aggregated output tokens maximum. But on the other side, we still need to think about the trade-off, which is the downside of offloading to DRAM, which is longer latency, especially for long-tail latency. So I guess the thing here is that we will customize based on different So for example, we've introduced another inference engine, TAO-RT, in our article where it's extremely focusing on low latency cases. Essentially what it does is that it has a mega kernel for decode to reduce the launch overhead. And in smaller batch cases, such like batch size equal to 1, it just shipped with blazing fast inference. decode speed, which is up to 470 tokens per second. And the trade-off is that it cannot serve with large batch sizes. But we've seen that more and more inference engines are trying to serve with this special use cases for general GPU rather than Cerebras. And people who want high interactivity and low latency will definitely love it. And I guess the trade-off really depends on how much of the cost you're willing to pay and what kind of the use cases you're dealing. If you are Jane Street, you definitely want your request gets like several seconds to get the output, right? But if you're just like users like me, I'm dealing with the normal agent tasks, I can just wait. I can just like maximize the output as long as the cost is affordable. So.

You're not using ultra fast mode, Wenyao?

No, I'm not using it. It's too expensive. Also, it's hard to, it's hard for me to feel the difference somehow.

Yeah. Kimbo, how about you? Are you using ultra fast on any of the coding agents right now?

Not really. I don't think I can even use fast right now. So yeah.

Okay. So what do you think about TylerRT and the trade-offs there? Because like Wenyao said, you got to pay significantly more for that. Faster token speed. It's a fun engineering problem, right? But maybe not a thing that is realistic.

My interpretation of, so back to like Jordan, back to, you were talking about the Ruben D spec, keeping memory capacity, keeping memory bandwidth, prioritizing that over memory capacity. I think that's a good example of all the design decisions that we saw as well, that memory Bandwidth is more important than capacity. So concretely, the way to think about it is, like we said, sparse attention doesn't help with memory capacity, and you will still need memory bandwidth to sustain the performance that you would like. And I guess in the ultimate speed-of-light performance, which we can see in some sense, we can see it in TileRT or the ultra-low latency regime is that you're fighting against how fast you can load your model or how fast you can load the relevant data to finish the very small, relatively small amount of computation you have. So in the ultra-low latency regime, memory bandwidth is the most important part. And the problem with memory capacity is that we still need a lot of them, but the, I think Dealing with insufficient memory capacity is a very standard problem in, I think, computer architecture. That's why we have cache hierarchy. And there are numerous ways of, through algorithm to hardware, through hardware to overlap the computation with lower tier, lower bandwidth memory transfers. So in Ruben D-Spec's case, I think they're betting keeping the memory bandwidth While trying to connect GPUs through networking and again, try to overlap them.

Yeah. Do you think if you had to forecast what Zed AI is thinking about right now, when they keep the model architecture the same for 5.2 to 5.3 and get such a performance improvement strictly from their post-training pipeline, in my view, do you think they're experimenting with even more efficiencies on future attention architectures? Or do you think they're just going to double down on what TSA and just keep it the same for some future You know, GLM-6 release or whatever comes next?

I think my guess is that they would probably continue. My guess is probably continue the same architecture, probably pushing further for 5.x or like 5.4 maybe. But based on their release, it seems like they, they will continue trying different architecture because not too long after 5.3, actually. They released 5.3 Flash, which is not really a flash model. It's, in my opinion, almost like a Frankenstein model where they do KDA, the Kimi Delta Attention, with DSA, as far as I remember. So they're mixing all the efficient alternative attention mechanisms, putting into a model and actually trained it. And actually the model is pretty good. So I think that The success of that model should be able— would probably translate into like a scaled-up version of a model with even better capabilities and having a similar architecture.

Yeah. Let's talk about the post-training pipeline. I mean, maybe as a bit of a backdrop, actually before that, we can talk about the capabilities, but the first time I remember people talking about this was, I'll show a tweet from Tuer Taxes, which then got Elon to respond, which then got Jia Tang, the founder of ZAI, to respond. You know, when do you think China's going to get a Mythos-class model? This is back in June when GLM 5.2 had been out for a little while. Elon responds, probably Q1. And Jia Tang says, won't take that long. I made a comment on, I made a comment on one of the episodes we recorded yesterday where I said to Dan, to me, GLM 5.3 is a frontier cybersecurity model because I get classified out of using the Anthropic or the OpenAI models, even though we're in the CVP red teaming program and we're doing stuff with the NeoClouds to help them do this, because it's not even the classifiers, it's the stuff trained into the model that I can't get around. And so I fall back to using GLM 5.3 and I need to be more creative in my prompting if I want to get the frontier models to not refuse me and pollute the whole context window where you have to like rewind if you want to try and do anything again. So. Anyway, all of this is accomplished on, I can show benchmarks and whatever, but like it's an incredibly capable coding model, especially for cybersecurity tasks. And it's open available. Anybody can download the weights. This is not the biggest model on the market. And all of the improvements from 5.2 to 5.3 were done through post-training. So, I mean, I kind of agree, just keep post-training going, but what are the unique aspects of their post-training pipeline that you think are letting them do this?

Yeah. So yeah, just to comment on the The tweet, I think J. Tang, the CEO, is pretty pretty vindicated, right? Because it took only like a month, I think, after that. So can't believe Elon got out Elon by someone. So like out Elon timeline. The only thing Elon is well not the only thing, but the thing that Elon is most known for for aggressive timeline.

Yeah.

But but but yeah. So. I think, I think it's beyond 5.3.

This is Mythos class right now, but it's, or Fable class, but it's certainly like approaching there. It's nipping on their heels pretty, getting pretty close, I would say. Right?

Yeah, true. So technically it's definitely not Mythos class, but I think it's still non-trivial to get Anthropic themselves to admit that 5.3 is, what do they call it? Fable class, I think. Fable, near Fable level.

Let's go find it. I'll go find that blog post from them.

Yeah. So yeah, it's, I think it's a non-trivial feat all within like a couple months of post-training on the same model. And yeah, so the, in terms of post-training pipeline, I would say among all the Chinese model labs, I think ZAI is the one that invests most heavily on post-training and focus, really laser-focused on post-training. As you can see from other model labs where they change the architecture every, like every once in a while, GLM has been focused on this same architecture starting from GLM-5. Despite, you see DeepSeek changing already 2 iterations of the sensor DSA, For the past year, keep releasing new versions of their attention and doing scale-up experiments. You say Antling doing all kinds of linear attention as well. StepFun doing also very cool things. MeMo being even more conservative on that side. GLM has stuck with the architecture mostly while focusing on post-training. And I would say the techniques in general aren't too far off. The basic techniques of their post-training aren't too far off from actually the DeepSeek v3. which is still GRPO-based. But I think GLM, the post-training pipeline, they have very clear steps and reasoning to how they adapt their algorithm along with the efficiency they need. So for example, the first step, they use a typical GRPO for general post-training. The first step, they do it in a synchronous version, in a synchronous way so that they could get the most stability out of the training. So that will be, yes, the first part. And then after that, because they would need to further extract the system efficiency out of training, and also because agentic workloads, they are typically longer in the sequence length, and GRPO has always I guess RLVR in general has always had the weak training signal issue. So as your sequence gets longer, the reward signal, the strength of the reward signal gets diluted through the sequence, roughly speaking. So they, because of that, they, the second stage, they changed to a more REINFORCE-like, slightly different variant of post-training algorithm where you could tolerate other issues of the system. For example, like policy staleness, so that you could use an older version of the policy, or you could use just the most recent version of the policy instead of having to deal with a lot more ratio policy staleness ratio related issues during calculation.

Yeah. So give me a quick summary. This is algorithmic related decisions that you're describing here. These are maybe unique decisions that they've made, but nothing I've heard you say, you know, before this talk or now would lead me to believe that GLM 5.3 is doing something so unique on the algorithm that it's so completely different to any of the other labs that they have a big advantage on in terms of like algorithmic work in the post-training pipeline. Is that fair to say?

Yeah, I guess what I've said so far is not too unique to the Yes, that is correct.

Okay.

Actually, I want to say what I feel about GLM 5.3 when it first came out, like to do with the— no, it's completely based on the post-training with the previous models. I feel like this is like a wake-up call for me. So if you see the benchmark results of GLM 5.3, it's gigantic results on the benchmark was improved more than 10% by just doing pulse training. And it's more GPU efficient if you're doing pulse training than pre-training a model from scratch. And I think that's a wake-up call for a lot of other companies as well, that they don't have enough budget to train a model from scratch, but they also want to improve their company's workflow for some specific area, and they happen to have that proprietary and private data, so they can just do post-training themselves and achieve a better result than some of the closed-source models. If I remember correctly, that GLM-543 was focusing on agentic reasoning and general RL, and we see all the 3 areas has improved quite a lot by looking through the benchmarking results. So that's like the day that I saw the Twitter where GGL 5.3 came out and all the benchmark improvement. I was like, no way, just done this by post-training. That's quite, quite amazing to me.

Yeah. So yeah.

Yeah.

Go, Kim.

Yeah, I totally agree with Wenyao on how, just how, how much juice there's left to squeeze by just post-training. And yeah, 5.3's post-training effort is definitely very impressive and Also cybersecurity. 5.2, in our article, I think we also compared GLM 5.3's cyber capabilities with 5.2. And it's like very notable. It's like a night and day capability difference. And that's all just post-training. So, so yeah, the part unique about 5.3, I would say, to be fully honest, GLM hasn't released like a detailed technical report since GLM 5. So working based off of talking with researchers and the blog announcements, I think there are 2 things that are most notable. One is the one that actually published results about or research about the— they're doing PPO-style training. So they're reverting in some sense, reverting back to the typical RL training format, having like a critic model, grading rollouts along the way, and Co-training the critic model along with the actor, the main model. So that's— they call it Single Rollout Asynchronous Optimization, SAL. And the second thing is they, in their public speeches and in the conferences, they talk a lot about— thank you for putting down the meme, by the way. Take credit. They talk a lot about— thank you. I don't know. It's, it's, uh, probably too Taiwanese, uh, too, too local. Like even my like Chinese friends, um, actually like the, the Slime authors and just Chinese community, they don't even understand that meme, unfortunately.

Maybe explain that a bit, Kimbo. It's, it's so funny.

Wing Yao, have you watched SAO, Sword Arts Online?

No, no, I haven't, but, but I, I've saw your explanation on Slack. That was, uh, that was fun.

Did they—

I explained it. I thought I— I don't know. Okay.

Do they understand this meme, Kimbo? 10, 15, something

like that? I don't know. No one— I haven't asked anyone about that yet, but—

Okay. But I jumped in earlier and I said something that might get misconstrued here, which is like they didn't have necessarily super unique advancements in algorithms. And you just said the word slime, which I was waiting for. The, both the single rollout asynchronous optimization, but then also all the infrastructure seems to me like work related to the efficiency of the training that just allows them to run a bunch more experiments, potentially much more jobs, potentially just like much more compute efficient jobs, which is itself an advancement that sets them ahead of anybody else with like equivalent compute or even more compute who just like can't do as much research as them.

Yeah, yeah, definitely. SLIME is highly regarded in the In the community, the open source community, they're probably arguably one of the best, if not the best design, architecture design of a post-training framework. And yeah, the efficiency, I'm sure they have like very good infrastructure to support that. They also, GLM-ZI, they also released an article, I think probably not too long ago, 2 weeks ago or something, about how they use their models to co- kind of collaborate with models to further help them improve the system efficiency. So that includes bringing up the verifying the GPU kernels for their crazy GLM 5.3 flash model on an unknown hardware. So on the, I think it's the, I'm not sure what GPU exactly, what hardware exactly, but not the GPUs. So they have unknown hardware. With different architecture, they use their models to improve that, and they have ways to improve the loop of improvement, and also monitoring system-level jobs, how to further squeeze out more efficiency out of the whole infrastructure. So yeah, I think their SLIME is definitely algorithm-wise, SAL is a clear innovation, which is applied in 5.2 and 5.3. And then infrastructure based on even just on what they open sourced, it's a very good improvement and definitely would help with—

And let me throw one thing in here that I think a lot of people have missed, or is at least something I've been focused on. If we go back to the GLM-5 paper. I want to read a paragraph from this. Uh, and remember, this is like almost a year old at this point. Um, they say, last, one more technical contribution lies in the fact that from the first day, GLM-5 is full-stack adapted to Chinese GPU ecosystems. We have successfully completed deep optimization spanning from underlying kernels to upper-level inference frameworks across 7 mainstream domestic chip platforms, including Huawei Ascend, More threads, Highgon, Cambricon, Kunlunxin, MetaX, and Enflame. When you're talking about the model helping bring up the infrastructure for serving GLM-5, it's not necessarily the case that this is just NVIDIA GPUs and kernels and the serving stack that we're talking about for SG Lang on the AMD or NVIDIA GPUs that we test in InferenceX. They're serving this model, at least they claim. For many months on 7 different domestic Chinese accelerators. What do you guys make of that?

First of all, they also publicly claimed that 5.3 Flash, the, when it's under the name AuxAlpha, they're fully served on Chinese hardware. So that's because I remember when people tested it, they saw AuxAlpha being very good while also being able to serve, I think, 100 trillion or something. Like a ridiculous amount of tokens compared to, I guess, based on the amount of tokens that they can serve for free, people thought that it's impossible that it's served on a GPU by a Chinese app because Chinese apps aren't that GPU rich enough, but turns out they're using Chinese hardware. So I think that assuming everyone's honest, I think it's— they're able to serve it. Like pump, just plumb through all the, the blockers, like the, the basics of the, the kernels, the systems, the plumb through everything. And, um, while serving, um, those models, um, efficiently.

Yeah. Here's, I've got on screen, OxAlpha was a stealth model on OpenRouter. Uh, it was only, uh, a month and a half ago, August 20th, when this happened. I, it feels like,

yes.

A while ago now, much longer, but anyway, things move fast.

They did bring up, I think, I think 1 month or 2, I think. So the 5.3 Flash is because, yeah, they released 5.3 on early August and then they released 5.3 Flash in late August. So that means that even if they decided to, that the problem is when they start, when they start developing the 5.3 Flash model. If we have to like kind of upper bound, lower bound, it probably won't be too long. It's more like 2 months or 3 months as far as I can tell, because betting on KDA is still a big bet before Kimi K3 comes out because no one really knows whether KDA scales that well. And KDA, I think Kimi K3 came out like mid-July. Yeah. Probably the most aggressive timeline would be they trained 5.3 Flash within 2 months. If that's true, that's actually pretty crazy. Yeah, it is really crazy. One thing I want to add about the hardware side of things is that I think I also wrote it in the article. We can kind of guess. So first of all, the arithmetic intensity of the attention operation during decode is typically matched with the arithmetic intensity of the hardware. So this is shown most notably in DeepSeek V3, which DeepSeek openly talks about it. So that being said, there is some sort of hardware and software co-design in this sense where you try to change the model, the configurations of the model to fit the hardware properties of the GPUs that you serve. So DeepSeek fitted for H800s. But then, so it got me curious and then I tried to calculate the number for GLM-5. One of the most interesting changes they did is they changed the number of heads. And also I can't remember the exact configurations they changed, but because of the changes, the arithmetic intensity is very different from the hardware, the arithmetic intensity for the NVIDIA GPUs.

Yeah, I'll bring this up here. Yeah.

Yes, thank you. So I think the calculation I did was somewhat around 120. for GLM-5 or the 5.x series models. And so if I, and then I did some like bare basic calculations, it somewhat fits with the more threads GPUs. So it's somewhere there. That does not prove that they're optimizing it. That does not prove that ZAI is actually serving models on those. hardware, it doesn't really prove that it's optimized for that, but I think at least it shows. My conspiracy theory is that they're probably optimized for it, but I couldn't get a confirmation. So Zhiyi, if you're watching this, I would love your thoughts.

We got high alpha here.

Yeah, but at least I think what we can— yeah, it's just calculate the seismic intensity, right? But at least I think it shows that they're definitely optimizing for something else. Or they're sacrificing quite a lot of software hardware optimization for other reasons that are not shown in public because it obviously wouldn't work well on, not the optimal for, let's say B200s or H200s even.

Yeah. It's a sign, I guess, like it's a sign of what's to come that somebody might make a software level co-design decision based on hardware that is not NVIDIA GPUs. I think that's quite a big statement.

Yeah. And I guess like adding on that, that with the capability improve of the models, we began to see that the R&D cycle has been shortened drastically, right? And take a kernel side as an example. Nowadays, agents, they can, with the proper tools like Kernel Development Agent or the CAKE just released recently, with the right harness and the right tools, People get to write kernels that perform much better than humans themselves. And we also got like leaderboards that agent writes kernels. As long as you have the budget, it will write faster kernel than humans. And actually it's like pretty interesting to see how different model companies going to, you know, doing compete with each other with the help of the agent. So it's like human plus AI to do the compute.

Yeah. How much, how many tokens do you think the AMD TileRT mega kernel for GLM 5.3 cost them to produce?

Must be a lot, I think. What do you think, Kimbo?

I don't have a good idea, honestly. I wonder, yeah, maybe we should ask.

Yeah, maybe we should.

Coming up with good ideas here. Okay guys, anything you think that we've missed talking about this? I think we went through InferentX performance results, some high-level discussion about the cyber capabilities of this model, what we're using it for. High sparse, high cache, you know, KB offloading, model architecture, post-training pipeline. What else is left? Rumors and vibes on what's to come with GLM 5.4? What do you want to speculate as we sign off here?

Yeah, I think we briefly talked about it. The 5.3 flash is definitely a success for them. They're able to mix DSA with KDA. So while like a very new architecture while still having a very great model, having great capabilities. So I suspect they can definitely scale up 5.3 Flash, the model architecture of 5.3 Flash to a 5.3 level size and probably even squeeze out even more performance or model capabilities. Yeah, I think 5.3 Flash has that crazy efficiency as well, right? Because they're linear attention and sparse attention. So yeah.

I'm curious about what they're going to do next because I think JOM, they definitely explore something else. By post-training, they show us like you can improve model capability, but not only scale up, but also squeeze up given your current parameter size. Even though you're not trillion model size, you can still perform as well. And for those companies who do inference for the model and they're heavily sensitive to the cost, I think it will be definitely a good call for them. Actually, we've seen like in the Bay Area, like Harvey has their own post-trained agent based on KIMI model. We'll also see Periodic Lab, maybe Kimbo, you can extend that, has done the post-training on AI for science. So I'm really excited to see how this trend will go and especially what GLM 5.4 will bring us.

Yeah. One, I guess one last thing I think about post-training that is very important, but not very much discussed, and ZAI also is not willing to clarify more either, is human effort. Last time in their public, like in conferences and things, they talked about they spent a lot of human effort into supposedly data when training 5.2 and 5.3. And the problem is what kind of human effort, what kind of, well, human effort, I would say, I'm pretty sure it's something related to data labeling, but the problem is what kind of data labeling and what kind of effort into data labeling. And yeah, that's one, like I would think it's also pretty important when it comes to post-training that they're able to get quality data.

Makes sense. So what do you think comes next? Do you think we get SpaceBunny Alpha is MiniMax and 3.1 Flash. Do you think it's GLM? You think KIMI comes out with something new? DeepSeek? We had a big release from Reflection. I don't know. What are you guys looking forward to most with the model releases coming out soon?

Expecting some smaller— I'm looking for some efficient model. Actually, Reflection's model is, it's really efficient. I'm thinking like maybe in the future we have large size model, right? $2 trillion, $3 trillion, and it's capable enough for people to do powerful stuff. But we also have models that offer good enough performance, but it's like extremely efficient that company won't care about its cost and still do some certain thing well, like reflection, that they released their models.

Makes sense, guys. Okay. Well, thanks for coming on. Whirlwind tour of everything GLM 5.3. And why we don't think model releases like this mean people should sell their memory stocks and why we expect Jevons paradox to continue to result in more people consuming a lot more tokens and a lot more HBM and a lot more bandwidth over time. Thanks for joining, guys. Great job.

Thank you, Jordan. Thank you, Wenyao.

Thank you, Jordan. Thanks, everybody.