Kevin Weil, OpenAI's Chief Product Officer, discusses how AI is changing product development. He covers the shift from traditional computing to large language models and the unique challenges of building products when the underlying technology evolves this fast.
Everywhere I've ever worked before this, you kind of know what technology you're building on. So you spend your time thinking about what problems are you solving, who are you building for, all those good product things. But the stuff that you're building on is fixed. You're talking about databases. I bet the database you used this year is probably 5% better than the database you used two years ago. But that's not true at all with AI. It's like every two months, computers can do something they've never been able to do before, and you need to completely think differently about what you're doing. There's like something fundamentally interesting about that. makes life fun here. Everything we're used to with computers is about giving a computer very defined inputs. If you look at Instagram, for example, there are buttons that do specific things and you know what they do. You're confident that if you do the same thing three times, you're gonna get the same output three times. LLMs are completely different than that, right? They're good at fuzzy, subtle inputs. You probably get spiritually the same answer for the same question, but it's certainly not the same set of words every time. fuzzier inputs and fuzzier outputs. And when you're building products, it really matters whether there's some use case that you're trying to build around. If the model gets it right 60% of the time, you build a very different product than if the model gets it right 95% of the time versus if the model gets it right 99.5% of the time. And so there's also something you have to get really into the weeds on your use case and the evals and things like that in order to understand the right kind of product to build. So that is just fundamentally different.
