Robert Ta and Speaker B argue that treating AI agents as intentional beings like humans requires oversimplifying what humans actually are. Neuroscience, psychology, and philosophy show we're far more complex than the intentional stance allows, making the comparison itself the problem.
Anything else you want to say on intentional stance or urging AI researchers to take a more mindful view of the disciplines of philosophy, epistemology, consciousness, et cetera?
Civilization is complex. Even like group coordination, like these edge conditions where civilization can end because of misunderstandings between nation states. It's really complex. And George has kind of like put this label onto the swarm of agents and called them a civilization. If you step back, what's the right point where just Saying this is a goal. I'm an engineer with a goal and I need to train a system to do something versus I need to use the intentional stance to describe agents and what's the right tool to use. The richness that we're even just talking about on the surface of what a civilization is, the intentional stance only works if you have a cartoonish version of a being or a human that you're considering. To say that an agent is equivalent to a human requires you to reduce the human to this cartoon-based version of what it is. As opposed to like a rich, evolutionarily informed, neuroscience, psychology informed point of view, like the one that Anil Seth presents. You're talking about like, okay, there's a cartoon over here, and here is like a mathematical conception of a beast machine over here, as presented by Anil Seth. And only if you stay in the cartoon world does the intentional stance make any sense to apply to agents. If you're applying like a real sense of what a human is, what a human civilization is, you're default reducing it to like this stick figure, this straw man, this cartoon. Of what it actually is and you're conflating it. So I think that's one thing. You're trying to sense-make this world, right? Like things are moving really fast. Where do you apply the lens of when and where intentional stance makes sense? Where do you apply intentionality to agents? Who defined the goal that the agents are following? Is it defined by a researcher in a lab who's like telling them what to do? These are things that you need to like make sense of the situation. And this is, I think, the test that Dwarkesh failed when he wrote the essay, is that he over-applied the tool of intentional stance and he under-applied the tool of human-controlled engineering, his overall point was, hey guys, we're losing control, or are we just not sophisticated enough at defining our goals for the control system yet? The whole essay kind of had this like askew premise.
