Carlo CocuzziAI Expert, Pod Cast Host, Real Estate Investor and Funding Specialist

"Ship of Theseus": How the Transformer Actually Gets Replaced

23 Aug 2026 · 1 min

Watch on YouTube

0:31

In this brief talk, the speaker argues that the era beyond the classic Transformer architecture has already arrived. He points to several emerging techniques—Mixture‑of‑Experts (MoEs), diffusion‑based Transformers, various linearized‑attention methods such as Moonshot’s approach, and the incorporation of recurrence—as the building blocks of a post‑Transformer model. Rather than a sudden, wholesale replacement, he likens the transition to the Ship of Theseus paradox: the original attention‑centric model will be incrementally dismantled and rebuilt piece by piece, resulting in a new architecture that retains continuity while shedding its old components. He predicts this gradual shift will be evident within the current year.

When do we get to something that's beyond transformers? I think we're there already. Yeah, this year for sure. We have MoEs, we have diffusion transformers, we have all sorts of attempts to linearize attention, including Moonshot's approach to linearized attention. We have attempts to inject recurrence into the architecture. So my bet is we get to the post-transformer architecture not through a step change, but through Ship of Theseus-style replacement of all of the individual elements of the original attention's all you need.