Enabling AI’s potential through Wafer-scale integration - Andrew Feldman (Cerebras Systems)

Name: Enabling AI’s potential through Wafer-scale integration - Andrew Feldman (Cerebras Systems)
Uploaded: 2019-09-17T16:50:03+00:00
Duration: 2 min 10 s
Channel: Andrew Feldman
Description: Andrew Feldman describes Cerebras's wafer-scale processor, which is 46,000 mm² and contains 400,000 cores and 18 GB of on-chip memory. Compared to the…

17 Sept 2019 · 2 min

Watch on YouTube

2:10

Andrew Feldman describes Cerebras's wafer-scale processor, which is 46,000 mm² and contains 400,000 cores and 18 GB of on-chip memory. Compared to the largest GPU at the time, it offers 78 times more cores, 3,000 times more on-chip memory, and 10,000 times more memory bandwidth.

Chapters

Not surprisingly, over the past 5 or 6 years, our industry has built bigger chips. Between 2013 and 2019, where we are today, the size of the chip used in artificial intelligence increased by 50%, just under 50%, 48%. It grew from 551 square millimeters to 815 square millimeters. More space, a bigger chip, allows you to put down more cores, allows you to put down more memory. We saw this, and at Cerebras we thought this was a good idea. So we built the biggest chip ever made. Instead of 815 square millimeters, we built a chip 46,000 square millimeters. Instead of having a billion transistors, we have 1.2 trillion transistors. Instead of having a few thousand cores, we have 400,000 cores, 18 gigabytes of memory, 9 petabytes of memory bandwidth. This goes on. This is actual size. Well, actually, it's just a little bit smaller than this. Um, this is a comparison of our chip and the largest GPU at market. This is one of my favorite slides, so we're gonna pause just for a minute. What does that mean in terms of these elements that we say drive performance? Well, 78 times more cores, 3,000 times more on-chip memory, 10,000 times more memory bandwidth, 133,000 times more fabric bandwidth.
That's what happens when you build really, really big chips.