Robert Ta argues that AI agents face a fundamental problem: handling subjective human requests like "what's for dinner" requires inferring what users actually want, not just answering literally. Unlike math problems with one right answer, these requests depend on individual beliefs and worldviews, making alignment and goal specification the core challenge.
If you're using AI from the spectrum of like, help me figure out a cooking recipe, booking a flight, be a therapist for me, then on the other end, it's like, yeah, I know we usually talk about help me build a mobile app, help me build this front end, help me build the server. There's trillions of things and intents that people have when they're engaging with one of these agents, whether they be ChatGPT or Claude Code or Claude or whatever. And every human is different and has their own evidence beliefs. For their worldviews. And that drives the intent, that drives the subjectivity of the goal. And there's this issue with goal in my head might not be goal that's stated cleanly out into the world, because there's the perspective taking that must happen in the world from your partner or from the agent to be able to meet you and align on the goal. So if we're having these problems, I know every company is having the problem. One of the analogies or metaphors I've been using towards Describing the problem. An AI product is not just 2+ 2= 4. 8 billion people in the world hopefully could agree 2+ 2= 4. But if you go ask 8 billion people in the world, hey, what do you want for dinner tonight? You're going to get quadrillions of answers. And so that's really the problem of an LLM-based product is what do you want for dinner kind of problem where it's non-deterministic by nature and it's really dependent on the subjectivity of the human. Just think about that one case. You ask a friend group, what do you guys want for dinner? All people will pop up and be like, hey, okay, maybe Thai food or Korean food or Mexican food. And then somebody will be like, oh, well, I had Mexican food 2 days ago. And somebody's like, oh, well, I don't really like Korean food, or I'm like a vegan. All of those things are what we mean by subjectivity in the individual that then leads to, if you map it out, the subjectivity of what do we inject into the context window? And then for clarity, our company, we're We're really thinking about that, like how do we model the self, the person, the human driving the agent to maximize their agency and also minimize the amount of turns they have to get to their goal?
