Andrew Feldman Interview with Cambrian AI Research

Hello, this is Karl Freund with Cambrian AI Research. Once again, I've got an industry luminary on the phone with me, or on the Zoom with me. This is Andrew Feldman, founder and CEO of Cerebras, the company that has made quite a stir in the industry with their wafer-scale engine. Some people say Andrew's crazy. Some people say he's visionary. I lean towards the latter, but let's let you decide for yourself. Andrew, welcome aboard.

Good to be here, Carl. Thanks for having me.

You bet. So Andrew, let's just start as I do with all of these interviews. Just describe your vision for AI. What role will it play in the world and the impact it will have, and then talk a little bit about Cerebras' role in achieving that vision.

Okay, I think we founded this company in early 2016 because we believed that AI would be a transformative technology in the 21st century. And we came to believe that it would impact the youngest of our society to the oldest. And that its ramifications would be absolutely across the economy and across how we work, live, and play. And I think we've begun to see this. And I think when a technology becomes truly ubiquitous, it sneaks in, and you don't even know you're using it, whether that's in your maps, right? I mean, you and I are old enough to remember when there were paper maps and when your Maps app didn't tell you that there was traffic there and didn't make recommendations based on the traffic patterns. When search wasn't putting up advertisements that were really useful, when drug design was absent AI and simulation was slow and painful, a lot of hit and miss. I think we also miss some of the simple things in AI. My mother-in-law is 88, and she asks Alexa to play Frank Sinatra, and the playlist that's provided includes songs she doesn't remember she knew. Right? And then she will begin to tell us stories about the first time she heard this song in 1946. What a lovely thing, right? And here's the simplest of AIs, um, and it's affecting and having an impact on the oldest in our society. And so across the board, we, we think this is a profound and important technology.

I couldn't agree more. I couldn't agree more. So talk a little bit about Cerebras's role. Talk about your company and how you're going to help change the world?

Well, we, uh, we set out to extend the reach of AI to the absolute hardest problems. We, we weren't interested in, in some of the simple problems. There are a lot of people that do that pretty well. Uh, we felt that the way to bring AI to a whole new class of problems was to rethink the, the compute machine that it operated on. And we began with the chip, and we looked very carefully at the, the characteristics of the workload, and we decided that for this work, a big chip was the right path forward. And everybody agrees, everybody over the past 14— since 2014, their chips have gotten bigger and bigger and bigger. And in the end, this was about the biggest chip that the industry delivered. This is an NVIDIA A100 chip. Now, we came along and said we could do better. This is our chip. And, you know, it's 56 times larger. It has thousands of times more cores. It has hundreds of times more on-chip memory, thousands of times more memory bandwidth, tens of thousands of times more fabric bandwidth. Andrew Feldman. And in our work, big chips process information more quickly, producing answers in less time. And we, we took this big chip and we built an entire system around it. And we began selling it to people who were working on the largest and hardest problems in society. So the very first customer is working on the largest cancer study. It's Argonne National Labs, and they've already tweeted about it and spoke about it. They're working on, with the NIH, a study that looks at the impact of drug interactions on tumor size. And with the, with the Cerebra CS-1, our offering, they were able to do work hundreds of times faster and find results that previously were unknown. And how rewarding is that for your first application? Uh, since then, customers including Lawrence Livermore National Labs and GlaxoSmithKline and Pittsburgh Center for Supercomputing have all announced solutions, and all of them are doing, uh, big things, right? Uh, at— one of the things they're working on at Lawrence Livermore is the study of traumatic brain injury among our warfighters. Um, at Pittsburgh Center for Supercomputing, dozens of researchers are bringing different problems to bear on the work there. At GlaxoSmithKline, their AI strategy is to cut down the amount of time it takes to identify and find a drug in half. And when you set out to be an entrepreneur, you hope that your work will produce this sort of impact. You hope that what you build will have a profound impact on society. In our first year of shipping, which was 2020, we were able to deliver to customers, and these customers were able to do things that had never been able to be done before. And that's something we're enormously proud of.

Yeah, I think that says a lot about what your company is all about. You're not trying to do what everybody else is doing. You're trying to do what nobody else is doing.

That's right. We are not. We're not trying to do inference in a phone or in a GoPro or at the consumer edge. We want the biggest and hardest workloads. We want to replace hundreds of thousands of GPUs. We want to do in training, and we want to do inference on satellite images and the most complex video, and, you know, be fed— Lawrence Livermore, they've got the 8th largest supercomputer in the world feeding us, and we're doing AI, right? They're doing simulation out of this machine, and the results of the simulation are being fed into us for learning. Now, that's exactly the way we train pilots, right? The pilot sits in a simulator, and the simulator generates scenarios, and the pilot learns from the scenarios. And with that, we've been able to, to make flying unbelievably safe. And we're doing the exact same thing with AI, where the supercomputer is generating the simulation and the scenario, and our system is being fed by this massive set of compute is doing the training. And these are things that hadn't been done before and are sort of pushing the boundaries in size and scope and in performance.

So let me ask you a bit of a technical question. You mentioned solving the really largest problems, and that's very exciting. It sounds like your customers are already doing that. But I have a conundrum. Conundrum?

Just a little conundrum?

Just a little conundrum. It's called memory. Something I'm not very good at at my age, but if you look at a GPU today, it's got 80 gigabytes of HBM2e memory. If you look at some of the startups, they've got a lot of SRAM, but then they augment that with DDR4 or GDDR5 or 6 to get really large amounts of memory to handle these very, very large models. How does your platform address the need of ever-increasing model size?

I think memory is not all of a piece. There's memory that's nearby and it's useful, and there's memory that's far away. And the way to think about it is, is memory on chip is like beer in your fridge when you're watching a ball game. It's super easy to get to. You don't have to interrupt anything. The game's still going on. You can get the beer and you can see. Off-chip memory is like beer at Safeway. All right, it's halftime, you're out of beer, everybody gets in their car and everybody at the cul-de-sac, all right, there's a little congestion at the cul-de-sac and they all drive to Safeway and there's congestion in the parking lot and they all run to the beer aisle, they grab the beer, there's congestion at the checkout, right? That's what memory off-chip is, is it doesn't help you go fast. If you want to go fast, if you want to keep watching the game, right, put beer in the fridge. And we've got the most beer in the fridge. And not by a little bit, but by orders of magnitude, right? You know, it's whatever, 400 times more than a GPU. And so that's what enables you to go fast. Now, if you want to go slow, I don't think you're doing big AI. And if you—

right?

And so, you know, that's the way to think about it. A lot of people are going to tell you, you know, I got a lot of beer at Costco. Well, it's a long way away to Costco. Right? And there's a lot of lines between here and Costco. And it's the same when, when all your cores— and when you have thousands of them, or hundreds of thousands like we have— are all trying to get to your memory at the same time. Alright, there's congestion. And this is why everybody who's ever trained a big model on a GPU complains about memory. And there's just nobody who's ever trained a big model— say a model that takes more than 32 GPUs, 16 GPUs— who said, you know what, this trains blisteringly fast, we're happy with this. Nobody ever says that, and you don't say that because memory is a fundamental bottleneck. And so that's the problem we solved, and we solved it by putting a huge amount of memory— you know, as I said, what, 450 times more memory than this sad little thing— 450 times in this. Wow.

All right, so sounds, sounds exciting. So you're in the fridge, Carl. It's in your beer— beer in the fridge. That's so easy to remember.

It's so easy to remember, man. I don't, I don't understand. It doesn't matter how How much beer is at Safeway? What matters is how much beer is in your fridge. Exactly.

I love it.

I love it.

So tell me about the software challenges and model porting challenges adopters of your platform have. And then we'll go on to one more question I've got for you.

Sure. I think in 2013 and 2014, it would have been very difficult to build dedicated hardware for AI. I think what happened was Google and Facebook, with respectively TensorFlow and PyTorch, what they wanted to do was invest in a framework, in a language for describing AI that was portable across hardware. I think having learned that they were stuck with Intel in the x86 world, what they were sure they didn't want to be is now stuck with NVIDIA in the GPU world. And so they built a, an upper layer language, all right, that allowed them to move their software work, their models across CPUs and graphics processing units and their own TPUs and later ARM machines, right, our CS-1s, and other startups as well. And what that did is it allowed us to disintermediate the proprietary stuff. All right, that all got pushed to the side. Nobody wants to write in a proprietary language. What they want to write in is TensorFlow or PyTorch or some sort of Python-based language. And what that allowed us to do was to, to develop a compiler that bolted on to TensorFlow. So right now, if, if you write a model in TensorFlow, our compiler takes it as is with no modifications and produces a config file for our machine and data streams to it. And so it— I wish we could take credit for that, but what, what it really was, was an industry saying we don't want to write low-level proprietary code and describe our models that way. What we want to do is we want to write it in an upper-level language, in a Python framework that allowed us complete transportability across different hardware types. Now it's not easy to write a compiler. And I'm sure that was part of your point. Um, we've been at it a long time. We have some of the best in the world at it. Um, and, uh, it's, it's working great. And our customers are finding that they deploy us and get us up to speed, uh, you know, 10 minutes after that system's rolled in and set up, models are training. And that, that's what you're looking for. What you're looking for is, is time to, to answer. And if each time you have a model, you need to rethink the learning rate, or you have to rethink the cabling, or you have to rethink your placement algorithms for work, right? That's not easy, right? You got to add that to the training time— weeks plus the hour and a half it took to train. What you want is to take two very different models, or two flavors of very similar models, Play, type \tf-run \cs1, and there you go.

Interesting. So I honestly was not surprised at all when you first announced clients, customers for CS1 were supercomputer centers. I've always thought the fastest way to simulate a phenomenon is to not simulate, but to estimate it. Based on training an AI with previously run simulations, right? It's roughly 1,000 times faster. So congratulations on those wins. That's awesome. And I'm sure those scientists are enjoying beating the crap out of your hardware.

As are we. As are we watching them. That's what we're looking for. It's the best experts in the world to beat the living daylights out of your hardware.

Yeah, awesome. So where do you go next? You mentioned GlaxoSmithKline, and that's great. GSK has done a lot of good drug discovery with supercomputing over the years. Where do you go next?

I think you often announce first with the supercomputing guys for two reasons. One, they're early adopters, clearly, but also they're quick to announce. When you have wins in large enterprise, and you may often have legions of lawyers that have that come in and there's a marching sound when their lawyers come in, sort of like the Romans. Phalanx of lawyers, I think, is the right description. And it takes a long time to march through the approval process for an announcement. That often takes months and months of time. Galaxa moved really fast because working with them, we were able to show really nifty results. And it's really an important part of their strategy. Going forward. But there are other enterprise wins. I think there are 4 segments. There's hyperscale, there's scientific compute or supercompute, there's large enterprise, and there's military intelligence. And we have customers in all segments, and business is booming.

Well, I certainly look to learn more about those clients when they're able to talk and they get past all their lawyers. Um, they're unleashed.

You know me, we, we, we love to talk about customers doing great things with our product.

Yeah, yeah, that's, that's, that's where it's at. That's where it's at. Well, this has been very informative. Thank you very much. Do you have any final thoughts to, uh, leave with the audience?

Congratulations on, on your new firm. Thank you. I think, uh, it's really exciting and brave to embark on a new adventure. I think AI is such an exciting place to be right now. And that's for all of us. That's for those doing traditional solutions and startups who are growing like crazy and solving new and interesting problems, all of which are, if we do this right, sort of make society better. And in big and little ways. And what a pleasure that is to be a part of.

Yeah, I couldn't agree more. I tried to retire, as you know, and failed miserably because I got so excited about what's happening with AMD.

I think retirement isn't right for us.

No, it's not our shtick, dude.

No, it's not our shtick. I think we want to be in the middle of things and building things.

Absolutely. Well, you're building an amazing platform, and I wish you all the success in continuing to do so. And we will be watching with great interest as you announce new technologies, and especially as you announce new clients and use cases. So thanks a lot for your time today. Take care, stay well, be well.

Cheers.

Cheers.