Ben Goertzel describes how the label "AGI" (Artificial General Intelligence) entered use: while editing a book (process began ~2001–2002; book published ~2005) he and colleagues (including Shane Legg, Peter Voss, Pei Wang, and references to Marcus Hutter) settled on the term for sociological clarity despite reservations. Goertzel critiques aspects of the words "artificial" and "general," notes prior uses of similar terms, and mentions Marcus Hutter's AIXI as a theoretical exemplar of ultra-general intelligence (but unachievable in practice). The tone mixes technical points with anecdote and humor.
Chapters
Maybe it's good to step back a little bit. I mean, we've been using the term AGI. People often cite you as the creator or at least the popularizer of the term AGI, artificial general intelligence. Can you tell the origin story of the term?
Sure. So, yeah, I would say I launched the term AGI upon the world for what it's worth, without ever fully being in love with the term. What happened is I was editing a book and this process started around 2001 or 2002, I think the book came out 2005. Funnily, I was editing a book which I provisionally was titling real AI. And I mean, the goal was to gather together fairly serious academic ish papers on the topic of making thinking machines that could really think in the sense like people can, or even more broadly than people can. Right. So then I was reaching out to other folks that I had encountered here or there who were interested in that, which included some other folks who I knew from the transhumanist and singularitarian world, like Peter Vos, who has a company, AGI Incorporated still in California, and included Shane Legg, who had worked for me at my company Webmind in New York in the late 90s, who by now has become rich and famous. He was one of the co founders of Google DeepMind. But at that time Shane was, I think he may have just started doing his PhD with Markus Hueter, who at that time hadn't yet published his book Universal AI, which sort of gives a mathematical foundation for artificial general intelligence. So I reached out to Shane and Marcus and Peter Vos and Pei Wang, who was another former employee of mine who had been Douglas Hofstadter's PhD student who had his own approach to AGI. And a bunch of some Russian folks reached out to these guys and they contributed papers for the book. But that was my provisional title. But I never loved it because in the end I was doing some what we would now call narrow AI as well, like applying machine learning to genomics data or chat data for sentiment analysis. I mean, that work is real and in a sense it's really AI. It's just a different kind of kind of AI. Ray Kurzweil wrote about narrow AI versus strong AI, but that seemed weird to me because first of all, narrow and strong are not antonyms. But secondly, strong AI was used in the cognitive science literature to mean the hypothesis that digital computer AIs could have true consciousness like human beings. So there was already a meaning to strong AI, which was complexly different, but related. We were tossing around on an email list whether what title it should be. And so we talked about Narrow AI, Broad AI, Wide AI, Narrow AI, General AI. And I think it was either Shane Legg or Peter Vos. On the private email discussion we had, he said, well, why don't we go with AGI, Artificial General Intelligence. And Pei Wang wanted to do Gaia General Artificial Intelligence, because in Chinese it goes in that order. But we figured gay wouldn't work in US culture at that time, right? So we went with the AGI. We used it for the title of that book. And part of Peter and Shane's reasoning was you have the G factor in psychology, which is iq General intelligence. Right. So you have a meaning of gi, General Intelligence in psychology. So then you're looking like artificial gi. So then that makes a lot of sense. Yeah, we use that for the title of the book. And so I think maybe both Shane and Peter think they invented the term. But then later, after the book was published, this guy Mark Gubrid came up to me and he's like, well, I published an essay with the term AGI in like 1997 or something. And so I'm just waiting for some Russian to come out and say they published that in 1953. I mean, that term is not dramatically innovative or anything. It's one of these obvious in hindsight things, which is also annoying in a way, because Josh Habakk, who you interviewed, is a close friend of mine. He likes the term synthetic intelligence, which I like much better, but it hasn't actually caught on, right? Because, I mean, artificial is a bit off to me, because artifice is like a tool or something. But not all AGIs are going to be tools. I mean, they may be now, but we're aiming toward making them agents rather than tools. And in a way, I don't like the distinction between artificial and natural because, I mean, we're part of nature also, and machines are part of nature. I mean, you can look at evolved versus engineered, but that's a different distinction. Then it should be engineered general intelligence, and then general. Well, if you look at Marcus Hutter's book universally, what he argues there is within the domain of computation theory, which is limited but interesting. So if you assume computable environments, the computable reward functions, then he articulates what would be a truly general intelligence, a system called aixi, which is quite beautiful.
Ixi.
I see. And that's the middle name of my latest child, actually.
What's the first name?
First name is Quarksy Q O R X I, which my wife came out with, but that's an acronym for Quantum Organized Rational Expanding Intelligence. And axi's the middle name is. His middle name is Xiphanes, actually, which means the former principle underlying aixi.
But in any case, you're giving Elon Musk a new child to run for.
Well, I did it first. He copied me with his this new freakish name. But now if I have another baby, I'm going to have to outdo him. It's become an arms race of weird geeky baby names. Yeah, we'll see what the babies think about it, right? Yeah, but I mean, my oldest son, Zarathustra loves his name and my daughter Shahzad loves her name. So so far, basically, if you give your kids weird names, they live up to it. Well, you're obliged to make the kids weird enough that they like the names.
Right.
It directs their upbringing in a certain way. But yeah, anyway, I mean, what Marcus showed in that book is that a truly general intelligence theoretically is possible, but would take infinite computing power. So then the artificial is a little off. The general is not really achievable within physics as we know it. And I mean physics as we know it may be limited, but it's what we have to work with now. Intelligence, infinitely general.
You mean like information processing, perspective?
Yeah, yeah. Intelligence is not very defined either. What does it mean? I mean, in AI now it's fashionable to look at it as maximizing an expected reward over the future, but that sort of definition is pathological in various ways. And my friend David Weinbaum, aka Weaver, he had a beautiful PhD thesis on open ended intelligence, trying to conceive intelligence in a.
Without a reward.
Yeah, he's just looking at it differently. He's looking at complex self organizing systems and looking at an intelligent system as being one that revises and grows and improves itself in conjunction with its environment, without necessarily there being one objective function it's trying to maximize. Although over certain intervals of time it may act as if it's optimizing a certain objective function. Very much Solaris from Stanislal Lum's novels.
Right.
So the point is, Artificial general and intelligence don't work. They're all bad. On the other hand, everyone knows what AI is. Yeah, and AGI seems immediately comprehensible to people with a technical background. So I think that the term has served as sociological function and now it's out there, everywhere, which, it baffles me.
It's like kfc. I mean, that's it. We're stuck with AGI. Probably for a very long time until AGI Systems take over and rename themselves.
Yeah, and I mean, and then we'll be stuck with GPUs, too, which mostly have nothing to do with graphics anymore.
I wonder what the AGI system will call us. Humans? I was maybe Grandpa. Gps.
