AI breaks the traditional SaaS model because GPU costs stay high at scale, unlike storage or compute. Nvidia A100 and H100 chips remain expensive on the margin. The speaker argues consumer devices are vastly underutilized supercomputers that could provide distributed compute power instead.
So a lot of the economics of SaaS and investment in SaaS are based on the insight that there's an economy of scale associated with having a whole lot running on said machines, right? You can just throw more storage, more compute, whatever, and it'll have declining marginal costs. One thing that's really interesting about AI is that doesn't seem to be as true anymore, that the cost of getting access to those A100s, those H100s, what have you, it doesn't seem to scale, that it just costs quite a bit on the margin. And that means you're not getting as good return on capital. At the same time, this device I'm holding right now is practically a supercomputer and is way underutilized in terms of how much, how many of those compute cycles are being used right now as this is recording a video. The opportunity to make better use of the MIPS that are now distributed in the hands of customers, I think may be the next frontier for how we're going to get better economics for compute.
