Robert Ta explains the intentional stance, Daniel Dennett's framework for predicting AI behavior by treating agents as rational entities with beliefs and desires. The core problem: aligning AI with human goals is hard because human language often expresses conflicting objectives, and we don't fully understand how goals form in the first place.
Intentional stance. I wanted to throw this out there. This is a term coined by the philosopher Daniel Dennett, who has great writing, by the way. It is part of a theory of mental content proposed by him, which provides the underpinnings of his later works on free will, consciousness, folk psychology, and evolution. So you can see how some of these topics intersect with what is a goal? What does it mean to be human? How do we see progress on these goals? And then if an agent's job is to align with the human, to understand its goals, even if the human's natural language could present tension and conflict or multiple competing goals, In the prompt. That's a very hard problem to solve. So we should ground it in the latest breakthroughs from experts on the topic of human subjectivity. So here's how it works. First, you decide to treat the object whose behavior is to be predicted as a rational agent. Then you figure out what beliefs that agent ought to have given its place in the world and its purpose. Then you figure out what desires it ought to have on the same considerations. And finally, you predict that this rational agent will act to further its goals in the light of its beliefs. A little practical reasoning from the chosen set of beliefs and desires will in most instances yield a decision about what the agent ought to do. That is what you predict the agent will do. And he takes that concept of intentionality from the work of German philosopher Franz Brentano when clarifying the distinction between mental phenomena and physical phenomena. Brentano argued that in contrast with physical phenomena, distinguishing characteristic of all mental phenomena was the reference to something as an object, a characteristic he called intentional inexistence. So this goes pretty deep, and this is just one little slice of this. And that's Jonathan's point is where this is an unsolved problem. How do you get these agents that are now being used by user consumer space, you know, individual users like kids, teenagers, adults to now businesses with decision makers with lots of budgets and high economic leverage participating in our society, which is capitalism. Whether you like it or not, that's what we live in. And they're deploying capital, making decisions with these agents. So the agent's job is to best understand the individual's goals, the team's goals, the company's goals, and organization's goals to help with those decisions to get to progress on those goals. So clearly there needs to be an understanding of the subjective goal space for a human and what it means to be human.
