Weiguo from SemiAnalysis interviews AWS representatives Rachel and another speaker about Amazon's custom AI chip Trainium 3, its performance and cost advantages over GPUs, and partnerships with Cerebras and NVIDIA. AWS claims 2 million GPUs in the cloud with plans to add 1 million more, serving customers like OpenAI and Anthropic.
Chapters
Okay. Hi, Karthik and Rachel. So this is Weiguo from SemiAnalysis, and then we have the 2 partners from AWS today with us. We're doing a sort of discussion on your partnership with NVIDIA, and then sort of your statement on the 1 million GPU deployment. Maybe you could elaborate more into that. And then lastly, on a discussion on training and the progress there and the partnership with Cerebras.
Okay, I can go first. Well, thanks. It's great to be here at GTC 2026 and thanks for having us here. Yeah, so we, I mean, our partnership with NVIDIA goes back almost 15 years. We were one of the first cloud providers to sort of start offering GPUs before when the GenAI was really a thing, right? So long partnership where we have deep partnership. I mean, we've been offering around like, if you look at our infrastructure today with NVIDIA GPUs, we offer about 2 million GPUs in the cloud, the largest cloud provider to offer 2 million GPUs in the cloud.
Yeah.
And we recently launched a blog stating that we're gonna and another 1 million GPUs just this calendar year.
Yeah.
So basically 50% of what our entire footprint we have built in the last 15 years. And I think that's sort of like a testament to 2 things. One, the huge customer demand we're seeing on AWS for this infrastructure, and also the testament to our ability to scale so fast and in such a short period of time. So I think that's what I need to say. We deeply value the NVIDIA partnership. And looking forward to sort of bringing the next generation of Rubin GPUs this year, other Rubin systems, and our customers are excited to have them.
So could you go more into like maybe on like what are the challenges to scale and how is AWS, you know, get better at that or what are your advantages there? And also when do you expect GooPin to come online?
Yeah, so I can start with that. So hi everyone, this is Rachel and I lead product marketing for AI infrastructure at AWS. So a common problem that we're seeing with customers is that it's easy to build a very cool demo on GenAI in days or even hours, as you can see from all these trade shows and industry events. But when customers really hit a wall is really when they deploy these AI solutions in production. It's not just about deploying the application on GPU instances, but it's also everything around it. Like the data pipeline, the cost control, and the security and compliance around it. So, and that's really when AWS can help with our customers with our over like 20 years of operating infrastructure at massive scale. So.
Yeah, I mean, we offer the largest, the broadest, and the deepest sort of services that customers need really to bring these applications to production versus doing a POC at a much smaller scale. Yeah, tying back to what Krishna was saying.
So let's go more into the Trainium side, right? So you have Trainium announcement at re:Invent last year.
That's right.
That was very— Trainium 3. Yeah, Trainium 3 announcement with a brand new scale-up architecture, right?
That's right.
So, you know, like, and then you recently announced a partnership with Cerebras to sort of do this disaggregated Prefilled decoding architecture. Could you elaborate more on that? And then how are your customers viewing these options and what sort of applications do you see Trainium bring to your customers?
Yeah, so I can go first, right? So one of the core strategic principles for AWS when it started 20 years ago was to bring down costs for customers to be able to access infrastructure. and also give them the wider selection of different hardware platforms available.
Yep.
And that's the reason why we have been investing in our custom silicon, going back to our Graviton generation of CPUs some 8, 9 years ago. And the same thing, that's why we're investing in Trainium as the AI custom silicon chips. We wanna bring down cost for our customers and also bring them the opportunity to have a broad selection where the capacity availability is not an issue for them to scale up. So from a cost perspective, I think our premium line of products offer 30 to 40% better price performance compared to the alternative accelerators available in the cloud. And that's, and also from the capacity availability, we free up customers who sort of like access to an alternative infrastructure, especially in this environment where the supply constraints both on the chip side and the power side and everything in between.
Yeah.
Customers appreciate and value that. And I think that's why we have invested heavily in Trainium. And Trainium 3 is our latest generation one, offering 2 to 3x better performance compared to the previous generation, Trainium 2. We plan to scale it to a million plus chips this year and more next year. And some of the recent marquee customers are OpenAI and obviously Anthropic has been using us heavily for training and inference. So we're sort of super excited about partnering with AI labs, but at the same time, I think we're also sort of like seeing a lot of interest from a broad set of customers, especially in the startup arena, looking at using Trainium for their workloads.
Is that for their chip development workloads?
Yeah.
Spectrum AI labs, that's—
Yeah, so in terms of the AI labs— Oh, AI labs. Yeah, the OpenAI and probably specifically, I mean, they are and will be using it for large-scale training and distributed inference workloads. And recently the announcement of the sort of Bres and the Trainium product with supporting disaggregated inference. And the theme is also similar there, like bringing down the cost for our customers, especially for inference, bringing down the dollar per token.
Yeah.
So that, I mean, that's something that we see customers sort of like tell us, hey, for us to deploy more GenAI applications, we want to see the cost come down. The partnership is sort of, is the primary goal is to try to bring down costs for our customers.
Okay, thank you. Yeah, I think, thank you very much for going through the efforts, you know, helping the ecosystem build and the efforts you're putting into the infrastructure and then, you know, offering the widest and the broadest range of products for customers. Yeah. Great.
