Why Positron AI is Choosing LPDDR over HBM for Next-Gen LLM | Researcher Conversations at GTC

Hello everyone, welcome back. This is Jordan Nanos here for Sail Media and SemiAnalysis at GTC 2026. I'm joined by Thomas Summers of Positron. We're gonna chat a little bit about everything that's new, right? You guys are moving on from current generation with the Archer FPGA and into next generation chip with Azimuth. And the Titan system. So yeah, welcome and excited to have this chat. Yeah.

Yep. Thanks for having me.

All right, let's start with a little bit of an update. You guys have had some big announcements and I have to say, like, for a chip startup, for a chip company to get to where you are right now, where people can actually use it both from an API perspective and then also people can, you know, in some cases log in and run benchmarks for themselves is, is pretty great, right? Haven't really raised that much money, haven't spent that much time actually doing development. You're already making real progress on serving tokens.

Yeah, it's, you know, been both wild, you know, company's almost 3 years old, but going from having, you know, first, you know, shipping pseudo-product, it's had a lot of bugs, but shipping our first product about 18 months in, and then now just under 3 years in having the announcement with Oracle is, I think unusual in the space to have that rapid of developments and real market success. But it's, I think, a credit to our team and sort of the unique approach we took to starting with FPGAs and then making that into a true product and not just a prototyping vehicle, but having that and having that rapid iteration cycle with customers has really enabled us to leapfrog folks that Started a lot earlier, had a lot more money, et cetera.

Yeah, cool. So can we talk about the feedback from customers? Like the big thing for me, at least in us at SemiAnalysis, where we use models and tokens is how you can pay more, you know, more per token, not just more for faster tokens right now, right? Opus 4.6 Fast is like the ratio is about 6 times more money for 2.5 times faster. So we're paying like a super linear amount. Do you think this is going to continue? Like we saw Codex Spark announcements as well, still research preview. Is that sort of the thesis of the whole Positron business, that it's going to be more expensive tokens but faster? Or do you think there's a way to also save money and serve faster tokens?

Yeah, I would say the Positron thesis is more aligned to trying to reduce the cost of tokens and still have them be high quality, fast, et cetera. I think while there's opportunity to differentiate on speed and there are other providers, or be it NVIDIA yourself, just reducing the batch size expected, that's how Anthropic is on all of their hardware platforms, delivering that higher level of performance. If you're paying 3 to 6 times more per token, they're willing to not cram that box with as many users. Anything from that approach to having architectures that are really solely designed for low batch, extremely high inference performance. But the problem for service providers is that the actual aggregate throughput of that is significantly lower. So it's not as profitable unless you can charge a significant premium above that sort of super linear. So, uh, for Positron, I'll say that, you know, we want to, you know, compete on both providing very high level of interactivity, like that tokens per second per user, that, that individual user experience speed, um, but actually have that be driving costs down from even the, the, you know, commodity, uh, you know, low to medium performance range that, uh, uh, is the bulk of the actual inference taking place out there.

Mm-hmm.

And then I would say just though that like using, you know, Opus 4.6 fast, uh, is addicting.

Yeah.

Like, uh, my, my personal sessions, once I, I switched that on, uh, I saw, you know, bill, uh, uh, you know, grow very quickly. But, uh, um, once you switch in that mode and have to then go back when you realize, oh, I don't actually need this result super fast, it, it is painful. So I, I see that value. Is it actually worth that huge price premium? Probably not in most cases, but, uh, from a user's individual experience, it is addicting.

Yeah. Yeah. Interesting. So, uh, maybe let's dig into the architecture a little bit. So you say there's, you know, a scenario where Positron is used to reduce cost, right? Let's say, uh, um, serve cheaper tokens in general. Uh, can you talk about exactly how that's happening on the chip itself?

Yeah. So, In terms of what we've publicly disclosed about our architecture, it is a systolic array-derived design, but really, really focused on maximizing memory bandwidth utilization. So on our FPGAs, we hit and sustain about 93% of the theoretical peak memory bandwidth. And what is the thing that is bound in decode or in inference overall, especially when you look at the attention mechanism itself, be it in prefill or decode? That's a fundamentally memory-bound matrix-vector operation. And so that, I would say, architecturally, we really focused on obviously getting as much of the bottlenecks between the memory and compute out of the way so we can achieve as much of that theoretical peak as possible. But then at the microarchitectural level, really treat matrix-vector-based math as a first-class citizen, which has been horribly neglected on NVIDIA, Google's TPUs, et cetera. We've actually had regression in that over recent generations. Going from Hopper to Blackwell actually had the ratio of your matrix-matrix, your GEM performance, that's when you go to matrix-vector, on Hopper it was a 16:1 ratio of matrix-matrix, matrix-vector, which is— your scaled dot product attention, that went to 32 to 1. So even though they increased the flops overall for the device, while also doubling the amount of silicon, they actually had the ratio get worse for the main constraint when you're scaling tokens. You have your attention mechanism growing quadratically in computational complexity, and they actually made that ratio worse. So our focus is really solving those bottlenecks, which have been neglected in the era of training. And when people are just focused on big, dense matrix-matrix workloads.

Yeah. And maybe just to be clear, are you guys talking about the ratio for your chip when compared to those?

1 to 1. So we have, yeah, so that's, we are going all the way to saying that, you know, we're, we're getting that same performance for matrix-matrix as matrix-vector. And, you know, we are sacrificing some elements to that. Like we, we, do have less total flops than an NVIDIA GPU, but we actually get to use those flops. And most importantly, the thing that is scaling the most in the compute, we get to use that orders of magnitude more efficiently and actual more throughput than NVIDIA and others.

Yeah. And can you talk about the high-level specs? I mean, others have made decisions at the high level on the chip to focus on, let's say, you know, SRAM or HBM or, you know, specific stuff in the actual logic. Is there a high-level summary that you can provide there?

Yeah. So with Osmov, our next generation chip, we're taking a pretty counterintuitive or just against the trend approach of actually leveraging LPDDR as the main memory technology. And with that, we're able to offer drastically higher memory capacity. So, you know, Osmov, we've announced we'll be shipping, you know, up to 2.3 terabytes of memory capacity per chip.

Mm-hmm.

And in about 400 watts TDP profile. So, you know, over an order of magnitude more memory than, you know, a B200 solution today. You know, even Rubin will be shipping with 288 to start, eventually go to 384. So we're in that 6 to 8x, you know, greater memory capacity capacity per chip while being less than a quarter of the power. And we really think that scaling that memory capacity is critical for longer context lengths, being able to support large models. When we take 4 of those OsmoV chips and put that in a Titan, which is the name of the server, that's designed— we want to be able to support 16 trillion parameter models running on a single box.

Yeah.

And think that If you are able to have that much memory capacity and that high realized memory bandwidth per server, you know, we still have great chip-to-chip networking. We've got 72 112-gig lanes dedicated for, for inter-chip communication. But you, you rely less on that if you can do more per chip. So we think, you know, when OsmoMOS is shipping next year, being able to support 16 trillion parameter models, millions of tokens context length is a very exciting differentiated capability.

Cool. All right. 16 trillion parameters, millions of tokens of context, good token efficiency, no HBM, no advanced packaging?

No advanced packaging, regular organic substrates. That's another big thing from a supply chain perspective. If we want to be able to support and deliver Hundreds of megawatts, gigawatts of compute capacity to the world, having the same supply chain that Nvidia, AMD, Google, all of the big guys are fighting over is a losing proposition as a startup. And so we want to embrace commodity memory packaging technologies, et cetera, and enable our advancements through architecture and through clever scaling.

Cool. Awesome. Well, I think that's a great place to wrap. Thanks for taking the time. I got lots more questions for you, but yeah, we're going to sign off on this one for now. Thanks everyone for watching. Thank you.