Hamel Husain argues that LLM privacy decisions can't be made alone. You have to weigh privacy against cost, speed, complexity, and model quality together. Self-hosting works but usually costs more and takes longer than using a provider.
Privacy doesn't live in a vacuum. There's privacy, cost, time to market, engineering complexity. All of these things are considered jointly. And there's also capability, how good your models are. Sometimes people can get carried away with privacy, say, oh, we need privacy, but they can sacrifice a lot of those other dimensions to the extent where it can be detrimental. So you have to be really clear on what all these things are jointly, like hosting your own LLMs is non-trivial. There's infrastructure that can do it, but it's non-trivial if you don't have lots of throughput and stuff like that. It's going to cost you. It probably will cost you more on average, like a lot more, because large model providers are able to aggregate over everybody and do things that you can't at like a small scale. People had similar concerns, I think, using the cloud. Is it really private? I think over time this group of considerations— time to market, engineering complexity, whatever— a lot more people went to the cloud. That's not to say that everyone went the cloud. There are some people that do not go to the cloud and they have legit reasons for that. They just can't let their data out of their data center. So you have to be really cognizant of all those dimensions.
