Dylan Patel explains how Cerebras fits 84 die-like tiles on a single wafer using vertical stitching to route around defects, creating a system with massive on-chip SRAM that matches an entire Nvidia NVL72 rack while cutting data-movement bottlenecks and power costs.
Chapters
So as you can see here, the Cerebras wafer, what they've done is they've patterned the wafer with 84 of these patterns, right, these die radicals. And their special sauce is they're able to stitch them together with a special method that is done in collaboration with TSMC, where the upper metal wiring layers, they are connected in such a way that not only are they like horizontally and vertically connected to each other, But they also have ways to route around defects, which helps them to yield the wafer. Otherwise, if they use the traditional method, a defect can— basically you have to throw away the whole wafer. So these 2 key innovations, right, the vertical stitching and the yield recovery, enables them to have such a huge amount of SRAM, huge amount of compute on a single wafer, on a single chip. And then the fundamental benefit of this is Of course, as you know, in AI we are extremely data movement bottlenecked. And so the most efficient way to transfer data is to just stay within the chip, right? Even your HBM or like your chiplet solutions over the CoWoS interposer, even that is still nowhere near as efficient as just staying within the die. So if you can have a huge die, then you can have a huge amount of compute without needing to transfer a lot of data in and out. So the analogy they like to give is that one wafer almost represents like a whole NVL72 rack, right? It's like having 84 chips in the same network and you don't have to move off the wafer. So huge savings in terms of communication power, and also you get a lot of benefits when you're able to parallelize a model across an entire wafer.