Cerebras Systems CEO Andrew Feldman on Scaling AI Innovation

Welcome to the Tech Arena Fireside Chat. I am Allison Klein, and today I am so delighted to be joined by Andrew Feldman, CEO of Cerebris. How are you doing today, Andrew?

I'm well, Allison. Thank you so much for having me.

I've had Cerebris on the show before, but I've never had a chance to talk to you before, and I'm very excited about it. Um, you know, you have had an incredible purview of Technology innovation from the data center to the edge. You've led multiple technology innovators during your time in the industry, and now you're the co-founder and one of, of one of the hottest AI silicon startups. What led you to decide to found Cerebras?

Well, Allison, my wife wanted me out of the house, and, uh, that was combined with the fact that a collection of extraordinary people who'd worked together at our last company, a company called C-Micro, were loose in the saddle and we began meeting. These turned out to be my co-founders here. We wrote two things on the whiteboard that we wanted to work together again and that we wanted to move an industry. And we care deeply about the compute industry and we wanted to put our shoulders to work and push up against the industry and see if we could, could move it forward. We saw the rise of AI, and this was late 2015, so it was early, and it, it wasn't obvious that it would be where it is today. This was before generative AI. What we saw was that this was a new and interesting workload, and that existing architectures didn't handle it particularly well. And that we felt that we could build a new type of processor and a new type of system that allowed AI practitioners to do work vastly more quickly with less effort and using less power. And that, that those were the founding principles behind Cerebras.

Now let's fast forward a decade. You have been grabbing headlines with stunning inference performance across a lot of models and We'll unpack that in detail, but I just want to start by talking about what drove you to deliver large-scale ASIC delivery. Can you talk about that strategy and how you've developed the product portfolio you've got today? Sure.

It's a, it's a good question. I, I think our view is that if you're going to attack an incumbent, doing something a little bit better and a little bit cheaper is unlikely to get the job done. And when we examined the, the problem of AI, it's very, very interesting. It is comprised of two elements for the computer: the actual calculations and the movement of intermediate results. And the actual calculations are trivial. There are a lot of them, but they're very, very simple. They're mostly multiply and add. The movement of intermediate results is an extremely challenging problem. You have to move intermediate results from memory to compute. And if you build a traditional GPU, you have to move these intermediate results among hundreds or thousands or tens of thousands of GPUs. So we thought that if we were able to build the biggest chip in the history of the computer industry, if we were able to build a chip 56 times larger than a GPU. We could move information a lot less. We could keep it on a piece of silicon. And when you move information on a piece of silicon, it moves at a thousandth the power draw. It moves at about a thousand times faster than if you have to leave the piece of silicon. And so we could keep this work in this extremely high-performance, low-power domain. If we built the biggest chip in the history of the industry. And that was what we, what we set out to do. And in 2019, we announced to the world that we had yielded, we're the first ever to yield a chip that takes the entire wafer. A chip that is 56 times larger than the largest other chip ever built. We're now on our third generation. Our first generation was 16 nanometer. Our second generation was 7 nanometer. What we're shipping to our customers right now is a 5 nanometer part. It is more than 46,000 square millimeters. It has more than 900,000 AI-optimized cores. So it's bigger than dozens of GPUs and it outperforms 50, 60, and in some cases hundreds of GPUs depending on the workload.

That's incredible. What was it like taking on the task of designing something that massive from just a pure engineering standpoint? And, you know, I, I know that we're going to get into some results, but like when it comes to the performance capabilities, how have you seen that manifest in terms of a superior solution?

Sure, it's a good question. You know, we We like to do fearless engineering. We like to attack problems that others haven't been able to or refused to solve. And the problem of wafer-scale compute had been tried many times in the industry and had always failed. And we were the first to succeed. I think those are the type of problems that excite us. You know, one of our founders said that, It's only interesting if other people say it can't be done. And that was an enormously motivating force, was that it had never been done before. It had been tried by people like Amdahl, right? Fathers of the field. And they were unable to get it done. And we did it. And it was the moment we saw a working wafer, it was extraordinary. We had done something that in the entire history of compute, no other team had ever done. And nothing feels better than that.

That's fantastic. Now, this— the field of generative AI, everybody knows it's moving at lightning pace. It doesn't seem like headlines come out on a given day without some sort of announcement of a new model or a new capability coming into the fore. And you have been doing an incredible job of delivering results across LLM models. Some recent examples are DeepSeq-R1, Mistral's La Chatte, Perplexity Sonar, and I'm sure I'm leaving some results, recent results off the table there. How are your engineers able to so quickly take a look at these new models, and update your performance characteristics and tune your performance characteristics for them.

They don't need sleep.

How we do it. Is that the solution?

I think we have been extremely proud at what our customers have been able to do with our machines. They've been able to deliver the world's fastest search, the world's fastest reasoning model, The world's fastest chat, just to name a few. And that makes us enormously proud. I think the architecture allows our engineers to move quickly. It's differentiated. It is not, doesn't look like a GPU. It doesn't behave like a GPU. And as a result, it wasn't constrained by the same weaknesses that a GPU is constrained by. A GPU is limited by memory bandwidth. It's an architectural limitation, a trade-off that was made way back at the beginning when graphics processing units were designed for graphics. And we made a set of different decisions. And the result is, you know, for something like R1, we are order 70 times faster than the fastest GPUs. And we encourage you to go to LeChat or go to Perplexity Sonar You use any one of our, of our customers' various models that are running on Cerebras, and the performance is dazzling and palpable. You get answers instantly, and it makes the use of generative AI much more interesting.

Now, I want to take a step back into your unique purview into this industry as CEO, and one of the challenges that you have is navigating the waters of model competition that seems to be evolving from Silicon Valley's battle with each other to geopolitical gamesmanship. Where is this going in your mind, and, and how does a company like, um, Cerebras navigate and ensure that you're delivering value to all your customers?

Okay, it's a hard problem, and, and one that until 4 or 5 years ago wasn't, I think, top of anybody's list. But right now we have, uh, extraordinary models being made by OpenAI and Anthropic in the U.S. and Cohere in, in Canada and Mistral in France. Uh, Falcon was one of the pioneering models built in the UAE. Our collaboration with G42 in the UAE has built a model called Jace, which is the premier English-Arabic model. I think the the drive to participate in this, in this industry, to be more than a consumer, is palpable among many nation states. And so I was just in Paris at the AI conference there, and, you know, the vice president showed up, and, and of the US, and, and you know, the Prime Minister of France and dignitaries from all over the world, um, talking about, you know, how to innovate better, uh, how to put in regulation that, that doesn't crush innovation. Um, I, I think the short answer is you, you negotiate through this carefully and thoughtfully with common sense. You try and do things that would make your mother proud. Um, I, I think Really, I think there's a lot of words, but if you do stuff that you're confident will stand scrutiny and that would stand scrutiny from your most difficult critics, your mom or your wife or your partner, and that you could stand behind, if you make decisions along those lines, I think they'll stand you in good stead.

Nice. Now let's get to the fun stuff. You know, unlike a lot of AI silicon startups, you have product in the market and you have for a while and you have customers deploying. Can you talk about key customer successes that really showcase what you're delivering and the kind of value that they were receiving? Sure.

Why don't we begin with one of our favorite applications? We partner with Mayo Clinic. And as you know, Mayo is perhaps the leading medical institution in the world. And together we designed and trained on Cerebras hardware a foundation model for personal— personalized medicine for genomics. And what this model allows you to predict is the rheumatoid arthritis drug that is most likely to be effective for you. And as someone who has rheumatoid arthritis sufferers in the family, what could be better than watching your technology and your team collaborate with the industry leaders to solve an important medical problem?

Sure. That's great.

We have done work with— with— Sandia National Laboratory, in which we were able to look at simulations of material behavior over time, and we were able to see further into the future in this simulation than any previous simulation. Right, how cool is that when your technology is being used to see into the future?

Mm-hmm. That's incredible.

We have done work with TotalEnergies the French supermajor energy company, in using AI and other techniques to model oil fields. We have done work with G42, our strategic partner in the United Arab Emirates, in Arabic language, making, making big language models available to 400 million Arabic speakers. We've also published recently models in Kazakh, and in Hindi, bringing the power of AI to underserved populations. We have thousands of users who are doing interesting things. The guys at LiveKit use our inference to build real-time voice agents. The GlaxoSmithKline has used our are modeled to train aides for, for researchers and able to, to consolidate and make more efficient their researchers. And the list just goes on and on.

That's fantastic. And I've, I've read about some of these in the news and on your site, and the, the results are stunning in terms of what this capability is delivering across industries. You know, as we look forward into 2025, a lot of people are saying generative AI inference is gonna just skyrocket across enterprises. I know you're talking to all of your existing customer base and potential new ones. What industries in your mind, um, will lead this adoption of broad proliferation of generative AI and what's that gonna look like?

I, I think, uh, you're asking for the crystal ball. I, I think we are, right now a wash in generative AI inference demand.

Mm-hmm.

I think it is inference, remember, is how we use AI, and training is how we make AI. And so once good AI is made, lots of people want to use it. And that's the part of the seesaw that we're in right now, right? The models put forth by the model makers, some in the open source community and some in the closed source community, are powerful and effective and useful. And as that's happened, more people want to use, they want to use more often, and they want to use for more reasons. And when that happens, you get exponential growth in demand for, for AI inference. And we're seeing this among consumers inside of departments in large companies, not just the marketing department. And, and I, I might ask you or your colleague, maybe a year ago you used generative AI once a day and now you're using it the way we use search.

Mm-hmm.

Right? Every hour that there's a, a query I think we're seeing it used to augment professionals by summarizing research literature, by doing research for them, by— we're seeing applications that are designed with agents to replace different organizational functions, HR, operations, designed to procurement, scheduling are all functions that I think over time are augmented significantly by AI of one form or another.

I think you're right. And it's one of the things at Tech Arena that we've really leaned into is just redefining marketing from an AI lens. And it's just one example. I am so excited to see this just flourish and unpack the stories of exactly what changes across all of the business functions and across all the business verticals. And I can't wait to make, um, Cerebras part of that story because I know that you're going to be there working with those industries. Um, I want to talk about one industry in particular. Um, we're at Mobile World Congress. Um, how is AI inference on the edge evolving? We, we relate your large chips to deployments in data centers, but they're deployed all over the place. How do you see the edge evolving and what is your role in terms of delivering the core capabilities that will be delivered at the edge in the coming years?

First, I, I think it is a tremendous inaccuracy to position edge compute against data center compute. I think all the data show that as devices on the edge increase their capabilities, they also drive increased demand for data center compute. And, uh, of course the, the iPhone is the, the quintessential example, right? That everybody was worried that it would mean less data center spend. And lo and behold, we, we make the, the device in your hand more powerful. It does more on the edge. And what do we want? We want more. And that means you gotta go back to the data center more and more frequently. And in fact, it has driven adoption of data center compute. I think it's the same with, with AI. There's rudimentary AI right now in handheld devices, in cameras, in cars. And I think far from reducing the demand for data center, they increase the demand for data center compute. I think the— there's a great deal of interesting work being done in very, very low-energy AI chips or subsystems that will go into edge devices. I, I think that's phenomenal. You should process as much data as you can right near where the data is being collected, right? Moving is expensive, right? So if you can process it or summarize it right near where it's collected, you are in the most efficient domain. And so I, I think that carriers have a tremendous opportunity. They have PoPs, they have COs, they have fiber, they have cell towers, all of which are going to be an important component in delivering AI to, to users. They have large enterprise users. They have the opportunity to stand up incredibly interesting services, new services to their users, and to use AI to fundamentally augment their business. I mean, wouldn't it be nice if your technician, when he or she showed up at your house, had a report from all previous technicians?

Mm-hmm. Yeah.

Right? Or that their routing algorithm routed them around traffic so they showed up on time. Yeah, right. Or that they, uh, or better yet, that after an analysis was done by an AI of the previous reasons they came to your house, they began with some hypotheses of what might be wrong this time. Oh, says the cable guy, you have a long run between the pole and your cable modem. This is a known problem. Guys have been out before. Here are some things that, that that are recommended for such situations. That way they begin a little smarter. Yeah, I think there's tremendous opportunity and carriers have that opportunity in front of them.

Andrew, you know, one final question for you. I know that you're speaking at MWC. Where can folks find your speech if they want to connect with you there? And also, where can they connect with you and the team online?

Well, we're at cerebris.ai. So you are always welcome to connect with us there. We have demos you can jump on. We have a playground. You can run various Llama models and other open source models, including R1. And we hope you enjoy that. We have a free tier and then several paid tiers and an enterprise tier. Jump on and play around and have some fun. I'm speaking on Monday on a panel led by Jay Prasad at Ernst Young, and we encourage you to show up and listen. We will be talking about agentic workflows in the large enterprise and the various opportunities they present to, to make our, our corporations more efficient.

Awesome. Well, thank you for having the audacity to design such an incredible chip, and thank you for your time on the Tech Arena today. It was a real pleasure getting to know you, and I am a huge fan of watching where Cerebras is going. I can't wait to learn more.

Allison, thank you and the, the Tech Arena team for having me on your show. Be well now.