Future of Science and Technology Q&A (July 19, 2024)

Hello, everyone. Welcome to another episode of Q and A about future of science and technology. I kind of feel bad that I'm leaving behind history and my general Q and A for kids and others. But the way the days have aligned, I'm back to the future again. So I'm always excited about the future. I have been ever since I was a kid and have spent most of my life trying to build aspects of it. All right, let's see what we've got here. So Anon is asking, Can AIs be creative? Should AIs rethink art? Well, you know, being creative is perhaps easier than one might imagine. I mean, if you just pick a number at random, it's a creative thing. Is it something where we care about what has been created? The fact that we got the number 572, which, you know, might be very significant in some way or another, but it's a creative number. The question is, is it a creative number that kind of attaches to things we care about? Now, when it comes to creating art and so on, one of the things I've done for many, many years is to sort of go out into the computational universe of possible programs, run those programs, and see what they do. What's interesting is, well, the first sort of big discovery that I made in the 1980s, early 1980s, was that even though the programs may be very simple, the behavior they produce may be very complicated. And this is typical of programs like cellular automata, where you have some line of cells and you just go down the page, determining the color of each cell from the colors of the cells above it based on some simple rule. Those kinds of systems often will generate extremely elaborate patterns, even from very simple rules. And they're patterns where, if you see the pattern, your immediate intuition will be there must be a very complicated way in which this pattern was produced. But the big sort of science discovery is that's not the case in the computational universe, even very simple programs can produce behavior that looks very complicated to us. Now, the next question is, is that complicated behavior art? Is it creative art? Well, over the last four decades or so, since I started doing these kinds of things, lots and lots of people have used those kinds of algorithmically generated forms to make what they consider to be art. And I think it's interesting because there's sort of this question of why does it seem sort of interesting as artistically interesting? You know, just the number 572 doesn't, in and of itself, typically seem to us artistically interesting. Yet these forms that are produced by cellular automata and so on, often do seem to us artistically interesting. I think one of the key points is that nature is sort of following simple computational rules in much the same way that our cellular automata or other simple programs are following simple computational rules. And we're very used to the forms we see in nature, many of which we consider to be beautiful, et cetera, et cetera, et cetera. But we're familiar with them and it's a kind of a metaphor that we appreciate. And so when we see kind of an artificial nature in these simple programs, that's something that kind of resonates with what we're already experienced with and seems to us to be something that is sort of connected to things we care about and so seems to us kind of potentially artistically connected. So there are plenty of simple programs though, where we look at it and we're like, I just don't understand what's going on. It doesn't look particularly rich and interesting to me. It just looks kind of complicated and kind of random. Of course, when we're generating images, it's very easy to make a highly creative image that consists of random pixels. That's just not something though, that we would normally consider to be artistically interesting. But it's also worth remembering that what is artistically interesting is something that is determined by our experience. So, you know, before people were, you know, painting sort of pictures of soup cans or random brush strokes or random or arrangements of colored, colored rectangles and so on, probably most of those things people would have said, oh, I don't know, that doesn't seem very interesting. But somehow the way that the culture developed, those became viewed as artistic and they had a. And that sort of became part of those, became something that produced a feeling in us that we found interesting. And I suppose that we can think about some of the role of art as producing a certain feeling in us. And the question of how we determine what that feeling is. It doesn't just depend on the pixels that we see in front of us. It will depend on other things that those pixels kind of resonate with in our minds. And so I think this sort of cultural ambience of the actual array of pixels is important. So now when it comes to AIs, well, they can go from generating random pixels, you can do that computationally, to generating things like cellular automaton images. Not particularly. Neural network based AIs aren't particularly good at that. That's something that's computationally very easy to do to Things where they're sort of generating interpolations of billions of pictures that the AIs have seen on the web. These are all kind of. It's sort of a continuum of different kinds of things, from the completely alien random array of pixels to the kind of. To the sort of algorithmically generated, sort of reminds one of, of nature by virtue of the fact that it comes from definite rules, as nature does, to the kind of human informed kind of side of AI. And, and what I. I wrote about this a while ago. If one looks at the kind of, sort of human informed AI image generation, it's very easy to go from the concepts that we know about, like pictures of cats and dogs and so on out into kind of interconcept space to things which are kind of reminiscent of those concepts, but aren't actually concepts that we humans have yet conceptualized, so to speak. And I think this question of sort of, what is the path of art in interconcept space? What is the historical path of art? An interesting question. I mean, there's a certain. Just as we can look at sort of the way that people have spread out and, you know, colonized the Earth or whatever, we can ask questions and that's sort of a process that's gone on where, you know, there were continents that didn't have people in them for a while, and then people got there and so on. And that's sort of a process of our species spreading out across the planet. So similarly, we can see this kind of interconcept space of potentially artistic forms. How have we gradually spread out in this interconcept space to kind of colonize that, to bring in things that we now think that now produce some sort of resonance in our minds that we think of as artistically interesting. And I haven't done that study. It would be an interesting thing to do to kind of see, see how that has kind of happened over the course of time as different kinds of imagery, for example, become things that sort of resonate with our minds. Now, you know, it's an interesting question whether AI can do things where one says, look, there's a thing that is reminiscent of something in our minds, but no artist has yet chosen to arrange things that way. But if we saw it that way, we would immediately recognize it. I think that's happened a bit with things like cellular automata, other simple programs that there are things where nobody would come up with that on their own. But once you see it, you somehow recognize it because of this resonance with what happens in the natural world. So that's. And I think there's sort of a question when you're producing artistic forms of both the core essence of what's being produced and the kind of dressing that it has, like it makes a difference whether the overall style is something that seems like a well drawn picture or whether it seems like a sloppily put together thing. And I think AI is pretty good at imitating the kind of the dressing of artistic forms. And that's something that probably helps. I mean, for us, again, it's a question of what resonates in our minds if we see something that is sort of drawn in a very childlike way with kind of big crayons and so on, that has a different. Even if the actual way the lines are set up, it has a very different resonance for us from something which is sort of elegantly done with an oil painting or something like this. Just as in graphic design, there will be things where sort of the details of the fonts and the arrangement and the spacing and the colors and so on have an immediate, oh, yeah, that looks like a 1980s kind of thing or something like that. Even if the actual sort of content in a sense is no different. The. Let's see, Let's see. Looking at a variety of. Of comments here, you know, Jay is commenting. What I think also matters is how creative the humans who write the code are. The thing that is really, I think, remarkable and was deeply surprising to me decades ago, and I keep being re surprised by it, is simple programs just out there in the computational universe produce the most amazing forms and ones that we cannot foresee, we don't expect. They're always doing things that are surprising to us, often things that are very aesthetic, but they're not things where we knew to make it that way. In other words, you just enumerate a thousand different programs and you make little pictures of them. I think it's almost the story of my life I end up with so often with. I've got some enumeration of some different kinds of simple programs, typically. I was literally doing this just last night for some particular project that I'm now working on about the foundations of machine learning, where it's like, here are lots of little images. Which ones look interesting and which ones should I use as illustrations for something? And the thing that is remarkable about that is even though I've seen this for, what is it now, 44 years, this phenomenon, I still get surprised by it. I can't say, oh, I wrote the code that I could foresee would produce this remarkable form. I just enumerated these possible programs and something that I completely didn't expect that turns out to be remarkable happened. So that's a phenomenon that is sort of a little bit of a different thing than what we normally imagine happening in art. I mean, in art it's much more of we imagine from the mind of the artist that sort of there is a direct path to the art that is created. It isn't so much the artist kind of points things in the right direction and then somehow computational processes or natural processes just create the form that we get. I mean, that's certainly in the splattering of paint, type of model of things. That is, the artist kind of just flicks the paintbrush and it's physics that determines the actual pattern of droplets. But that's not the common case. The common case in art is that we imagine the artist is going to go the whole distance and it's going to be a communication from the mind of the artist through the medium of the thing that they create, directly to the mind of the viewer or listener or whatever else. And this idea that there will be a step in the middle that is sort of a computational and an irreducible computational step that sort of creates something surprising and remarkable just in a purely computational way. That isn't something we usually imagine happening in the process of making art. And it's something that, well, I've seen it for four decades now, and it's an interesting thing. It is the same sort of origin of aesthetics, beauty, whatever one wants to call it, that happens in nature. That's, you know, nature doesn't have. We don't think of it as having sort of a mind that is communicating with our mind to deliver an artistic experience. Nature is just doing what nature does. I mean, unless one gets very sort of theological about it, it's not a. We don't normally think of it as a mind to mind transfer of experience. Let's see, Let's see. Sean is asking, do you think art is a kind of multimodal scale compression of very complex perceptions or ideas into a single form? Is art a way of coherently representing lots of unconscious computation? That's an interesting question. I think. You know, something I've been thinking about recently is the big invention of our species is language. Language is our way of taking the thoughts we have in our minds, packaging them up in a transportable form, and being able to say them to another mind that can unpack them. That's the big thing that our species has Invented this ability to, to take even abstract thoughts, package them up and communicate them. But actually, one thing we haven't been able to do is to take the mental images that we form. We don't have the kind of hardware to go from mental images to transfer them to other minds, to other people. I mean, we can do it by painstakingly drawing something, but it isn't like speaking where we're kind of packaging up those concepts. And there's a big sort of a. It's an easy thing for us to do. It takes effort and skill to sort of communicate things by drawing pictures and so on. Now, AIs have a somewhat different form of, kind of delivery mechanism. And so with AIs it is possible to take sort of the mental images of AIs and make them things that we can see and other AIs can see. So it's sort of an interesting new channel of communication. Generative art is sort of, in that sense, a new channel of communication where you can take the ongoing thoughts of a mind and convert them into images. And, you know, it's often said a picture is worth a thousand words. It's like, what would it be like if we were communicating to each other not in words, but in pictures? And how would that sort of affect the way that things work? I mean, language and the idea of being able to communicate abstract things from one person, one generation to the next is sort of a defining feature of the creation of our civilization and things like this. What would happen if we had this much sort of higher bandwidth sort of thought communication mechanism of, you know, each having our little immediate sort of thoughts to pictures transducer, so to speak, and where we could go, you know, thoughts to pictures, and then we can receive those pictures and then form our own mental images from those pictures. Right now, if I have some mental image, the only way I can communicate it to you all is, is quickly, at least just by saying, well, I see, you know, a red dot and a blue dot and a green thing and a line from here to there and so on. Maybe if I was a good sort of sketch artist, I could start, you know, drawing the thing out. But you know, that that's. That's as difficult for us typically as it is for, you know, a dog to be typing by putting its paws on big keys and so on. So the question is, if we were to imagine this world in which sort of our mental images are immediately communicable, what would that be like? I mean, for example, those mental images, we might. Would the forms of art that we can imagine. If we are immediately converting mental images to sort of concrete form. There are different kinds of art that we can imagine now in terms of what art is really communicating. I think this sort of actualization of mental images is a different form of taking the thoughts we have and actualizing them. When one has something like poetry, which I always find really hard to read, one is essentially, one is trying to, I think, communicate at a level that isn't just the standard selection of words for the sake of sort of communicating a particular sort of exposition of ideas. It's intended to be something that probes deeper in the sort of patterns of thought going on in the brain that was creating it. Sometimes it's really hard for the brain that receives it to unpack that. You know, I think the thing that is ultimately one could sort of imagine in terms of the communication, in terms of artistic communication, is what if you could take the raw thoughts of the artist and transfer them to the mind of the viewer, so to speak? Well, I. I don't think it makes sense to take raw thoughts and transfer them because all of our brains are set up differently. Our training of our brains is different. The actual arrangement of which neurons go where and so on is different. The actual sort of pattern of activations in our brains, we can't transfer that any more than if you have a computer that has certain bits arranged in its memory when it's running. You likely can't transfer anything but sort of trivial chunks of that directly, bit for bit, to another computer. What you can transfer is the things which are sort of packaged up into some kind of protocol or language or something, or even user interface. It's not sort of the raw bits underneath. And it's the same with our brains that the raw bits underneath we can't expect to transfer. So then the question is, well, how deep can you go in transferring those things? Can you go beyond the obvious packaging into words that we do with language? And probably the answer is yes. And perhaps that's what one's trying to get at to some extent with things like poetry. But it takes effort and skill, perhaps, to both. Well, certainly take skill to package that way. It also takes skill to unpackage, to unpack those kinds of things. So I think that's. And I don't know, in the case of, I don't know, something like painting or something. You know, one has the feeling that there's a little bit deeper. Or music, for that matter. There's kind of a deeper trawling of the mind of the creator than the sort of, in a sense, surface level of standard expository speaking, let's say. But those are. And if you ask in the case of kind of the AI, and you say, well, this picture that the AI made, there are probably clues to its deeper construction, the deeper construction of the AI in the particular arrangement of pixels that it used to sort of set up the whiskers of the cat or something. But it's not clear those things. We don't necessarily have any kind of visceral reaction to that detailed arrangement of pixels that made up the whiskers. That, in a sense, is a reflection of. Of some inner thought of the AI. It isn't something where we have managed to become familiar enough with that, that it's something that can get communicated and unpacked by us. Let's see, Looking at different kinds of. Fox is commenting. There are fundamental principles in art seen clearly throughout history. The question is, how much of these fundamentals does the user have a grasp on and how can they use that as leverage? Yeah, that's interesting. I mean, like, when you're trying to understand something, if. If I say something in English and I use a word that you've never heard, it's not very good communication. And it's the same thing. I think with art, there may be some device that people use. Oh, didn't you know that if the person was drawing, drawn, you know, holding a cat, it means that they're a. Or if they were drawn with glasses, it means they're an intellectual or something. And, you know, those are sort of clues, maybe cultural clues, maybe artistic clues, maybe just sort of an artistic code that says that. And if you don't know the code, just as if you don't know a word in English, if somebody says that word to you, you don't get that communication. And I can completely see that that's true in art. And I know it's sort of an interesting thing. To what extent do you need to know art history to appreciate art? And you could say the same thing. I mean, to what extent do you need to know language to appreciate what's some document that's written in that language? Clearly, if there are words you don't know, you're out of the game pretty much for that document. But I think it's the same thing when I don't know, if I read some paper about some subject where I kind of know a decent amount about that subject, it's a lot easier for me to appreciate what's being said than if it's just like, well, here are some words, and I kind of know what the words mean, but I don't really have a big grasp of the story. So I can't really fit what I'm reading into this whole sort of conceptual framework that I've developed. So I think it's true, and I'm not as well educated as I should be about art history, but I think it's true there as well, that to appreciate it, well, you need to know more of its kind of cultural ambience than just what are the pixels on the image, so to speak. Sean is asking, could there be laws of art available to science using AI? You know, people have wondered for a long time, are there kind of principles of aesthetics? Are there things that one can just describe, in a sense, scientifically that say one thing is aesthetic and another is not? I mean, if we go back to antiquity, the Pythagoreans were big on certain chords in music are harmonious and others are not. And if it's an octave, a factor of two in frequency, then it sounds good. If it's a perfect fifth, a factor of three halves in frequency, then it potentially sounds good. If it's a factor of square root of 2 in frequency, a tritone, it tends to sound bad. That was from very early times. That was kind of a mathematical principle of aesthetics. And do we understand that principle now from a. Of psychophysics point of view? Not completely. I mean, I suspect that part of the story there is that when signals are transmitted on the auditory nerve, that there are kind of. There's sort of a signaling. There's a way of doing signaling there that operates at sort of a few hertz, a few cycles per second. And when you are combining different frequencies, there's the question of whether the combination of frequencies winds up, kind of ends up producing beat notes, for example, that lie within the region that is sort of the signaling region, which sort of feels awkward to us because it's like, well, you're not really supposed to be using that for content. That's some kind of meta content, something like that. That that's. That that's sort of the explanation, but we don't really know this, that there's some kind of explanation for this original Pythagorean discovery that there are numerical features that make some things aesthetic in music and not others. So that was a case where very early in the history of science, people kind of felt that there would be kind of a scientific explanation for aesthetics. If we think about that at a visual level, There are. Well, for example, in the case of color, there are quite a lot of theories, although less well defined, for sort of which colors go with which other colors. That's something that was very popular about 100 years ago, or a little bit more sort of coming up with theories of those things and how do they relate to our red, green, blue, color receptors and so on. It's not really completely clear when it comes to other forms of, for example, visual aesthetics. There's certainly principles of graphic design, but often it's kind of, you know it when you see it. Now, in the case of. When in building Wolfram language, for example, one of the things we've had to do for three and a half decades is do what one might think of as computational aesthetics. We've got to figure out when you automatically produce this visualization. When you're producing something with curves wiggling around and labels and axes and all this kind of thing, what decisions should you make automatically? But you need to do it automatically because you're going to be throwing lots of different data to visualize what decisions should you make automatically that will make these things be aesthetic to people. And that's been an interesting process, and we've done lots of work on that over the years of trying to figure out what is it that makes this thing kind of makes this thing look good and that thing look bad. And I think we've done a pretty good job in general, but that's been a case where it's kind of like taking this kind of very human impression of aesthetics and being able to encode it algorithmically. Interesting point is that some aesthetics tends to evolve with time. Something which looked pretty good. Those colors looked good in the 1980s. They look kind of dated today. And in any given year, there'll always be the, you know, the color aficionados and the color companies will always be putting out. These are the new colors with all these funky names and so on. These are the colors for this year. And there's this strange flow over the course of time of the colors people prefer. There's a purple year. There's a year where things. There's a particular kind of blue that's really the. In blue and so on. And you see this in fashions and so on and for clothes and other kinds of things. I have to say I've not. And it would be an easy thing to do. I've not made the obvious kind of plots of if you look in color space, you know, how. How do us humans sort of migrate around in color Space one, you know, how do our preferences migrate in color space? But I think this idea, as I say, that people have talked about for a long time, certainly ever since the Pythagoreans, basically, that's a couple of thousand years of sort of what is the scientific basis of aesthetic experience is something where, like many things that have happened with the practicalities of computation, we've had to actually jump in and do something about that. Now there's the question of sort of, to what extent can AI help with defining those aesthetics? I think there are two issues. One is, can you make things which will be aesthetic? And the other is, can you have a narrative theory of what seems aesthetic? On the can you make things that seem aesthetic? There's sort of a huge advantage that sort of generative AI has because it's seen all the things that people have put out there, and if it just sort of imitates those, well, those things will seem familiar to people and they'll seem kind of comfortable and in many ways aesthetic. As I said before, I think one of the surprises is what happens sort of in the raw computational universe, where the aesthetics is probably resonating with the aesthetics of nature, which is sort of a different kind of thing. But it's worth remembering. I mean, when it comes to kind of the way that aesthetics works, there have been different periods, different things. People have believed, for example, One thing from 100 years ago, sort of modernism in all areas, a little bit more than 100 years ago, that led to kind of buildings that are pure cuboidal boxes and led to this sort of very minimalist kind of design for things, which I think, in the end, most people would say has not been resoundingly successful. There are better and worse ways to implement it. But the things that originally were the kind of, the decoration, the ornament, the. That gets put onto things, whether it's serifs on characters for fonts, or whether it's moldings in buildings or these kinds of things. These things have a certain value to us. They have a certain. And I don't know, I don't think that value is purely a value of familiarity. I don't think it's like we like those kind of forms because they've been around since ancient Greek times type thing. I think we like those kinds of forms for some more fundamental reason. And perhaps that fundamental reason is because they are somehow reminiscent of nature. They kind of make sense to us because they're what we've seen everywhere in nature. So rather than, oh, it's because it's what we've seen in civilization since Greek times or something. It's rather. It's what we are confronted with in nature forever, so to speak. Or perhaps it's something to do with sort of the construction of our brains. And I think sometimes it is like the example I gave of good and bad sounding chords in music that I suspect is some parts of that have to do with the actual sort of information encoding hardware construction of our brains. But I don't think that's as likely to be true when it comes to visual forms and so on. Again, you know, I. Yeah, I think. But it's an interesting question to what extent we can have a science of art. Well, you know, to take the other branch of can AIs tell us why this is a good or bad picture? I haven't actually tried that. I. I don't know how well a typical LLM will do, a multimodal LLM will do. If you showed a bad piece of graphic design and you say, well, what's wrong with this? I don't know how it will do. It's an interesting question. I suspect it might not do terribly. It might say, well, this isn't traditional, this spacing is bigger than it would normally be and that's something it can tell from. In effect though, this is not how it actually works. Kind of looking at this is an outlier relative to the statistics of other things that exist out there as examples of images. Sub comments AI art is already a form in itself. I'm usually able to tell AI art from human art, but maybe that will be harder as the technology progresses. Yeah, my guess is it will get somewhat harder. Although, you know, I think it is. One can absolutely tell typical AI art from typical human art. There are human artists who I think make things that look a lot more like AI art. So it's. And one of the sort of strange loops that can happen is, you know, people become so used to seeing AI art that they start concluding that's how we should make art and the human art becomes closer to AI art. But I do agree that there are aspects of kind of the sort of the finishing of images that does have a quintessentially kind of AI form. And I can't really put my finger on exactly what that is. I think there's a certain sort of smoothness to the way that things are put together that is perhaps more. It feels more like an interpolation between things, whereas the human art seems more like kind of a cutout of this and that. That are sort of separate thoughts, so to speak. Perhaps what's happening in the AI art is something more, a kind of emerging, an interpolation between elements, rather than sort of elements being separated as different artistic thoughts. I'm not sure if that's right, but that's an attempt to kind of verbalize my impression of what one notices in kind of AI art. And at some level, one can sort of take apart and critique AI art as saying, well, actually, when we have this picture of the cat and the dog together, or the cat riding on an elephant or something, it's really just taking elephant pictures and taking cat pictures and finding this way to sort of interpolate between them and sort of putting pixels that aren't obviously in the wrong place, so to speak, but that that's sort of a weak thing to do. And if a human were doing it, they would really conceptualize what would it be like for the cat to be sitting on the elephant. And, you know, here's how that would sort of. That would really look more so than just, well, we can interpolate the pixels. Let's see. Elsie is commenting on this transfer of images without language serialization. Do you foresee something similar for complex abstract ideas embodied in human neural nets or firing patterns? You know, what is an abstract idea? It's something where the only concretification of abstract ideas is in some kind of formalization, whether that's in human language or in mathematics or in computational language. But abstract ideas, we don't have a way to pin down an abstract idea except by kind of giving it these kind of coordinations, these concretifications in something like human language? So the question is, what are the forms of sort of formalization of ideas, of abstract ideas, of concretifying actualizing abstract ideas to the point where they're communicable? And as I say, one of the forms that I can imagine is the picture form. And by the way, I realize in a lot of work that I've done in science, that's exactly what I've done. It's like, rather than giving a whole bunch of descriptive words, it's just like, here's a picture. The computer made it. It's going straight from the mind of a computer to your mind. And it didn't go through my mind. I don't know what all those pixels are. I've never personally internalized all those pixels. I'm merely showing you the pixels that the computer generated that represents this idea, for example. Now, what else is there along those lines? Could we transmit the raw embedding vectors from the neural net to another mind? Could we learn. Could we learn to read QR codes, for example? Could we learn to take something. Let's say that instead of us having words about things, we would take the meaning vectors that we get from inside the AI, the lists of numbers that represent, for purposes of the operation of the AI, the kind of. The position and meaning space of the concept we're describing, and make that into a QR code. And it's like, this is what I'm saying. Here's a QR code. Would we be able to unpack that? My guess is we'd have a hard time. My guess is that's just not the architecture of our brains. One of the things that we have to recognize is there are things brains can do, and there are things brains find it really hard to do. And I think we're seeing that with LLMs as well. There are things that LLMs find fairly easy to do, and there are things that LLMs find really hard to do. For example, if I were trying to generate one of those, you know, cellular automaton images in my brain, in my brain, I wouldn't be able to do it. Not a chance. You know, 12 million operations that make little pixel dots all over a page. I don't do that in my brain. I'd never get that right. Even if. Even if I could follow it for a few steps, I'd never get those. Those 12 million pixels and so on. And similarly, if you say, run this program in your brain, one can't do that. It's just not something brains do. And if you say even things like, I don't know, take this graph that has all these connections and so on, so it looks like a big tangle of wool, and understand what's going on in it. It's not something brains typically do. So there are features that. There are things that brains, as they're currently configured, sort of have the hardware to do, and there are things where we don't have the hardware to do those things. And if we're going to deal with those things, we have to have sort of an attenuation of those things that we can actually fit into our minds. I mean, this is the whole story of a lot of things in nature and in sort of the idea of science, in a sense, is to take everything that happens in, for example, the natural world and somehow attenuate it, somehow sort of just extract those pieces that we can sort of have a narrative about in our finite minds. I've written a whole bunch about this in the context of what I call observer theory. And it's quite that that sort of process of compression, attenuation, equivalencing is very important process in terms of what we extract from the world and call science, so to speak. But, you know, I think this question of whether there are sort of different forms of communication, different things that are perhaps more direct brain to brain communication, it's an interesting question. I mean, it relates as well to things like direct brain interfaces, direct neural interfaces and so on. And sort of, I think, a complicated issue of what would it mean if we had sort of a direct neural connection? Does having a direct neural connection mean that we can immediately sort of superpower our brains to be able to think better, be smarter, et cetera? I think that's not obvious because I think that it is. We can certainly imagine that direct neural connection, which runs away from language whenever you think about some particular thing in a crisp enough computational way, just goes up and runs it and tells you back the answer. Well, that's certainly a thing to do, but it's not as dramatically different as type it with your fingers and see the display of what the results are. It doesn't feel. It is not highly integrated. It's very similar to what happens with kind of tool calls from LLMs. Tool calls are, in a sense, neural implants into an LLM. They are, you know, you take the neural character of the LLM and you connect it to this thing that might be, for example, Wolfram language or Wolfram Alpha or something like this, where you are going and, you know, the neural net has been going and thinking, it's thinking, and then it crispens that up into some kind of interface language that it can then send to the computational language. You then compute the answer and then send it back to the sort of neural net layer. And the question of whether sort of with sort of the direct neural implant, can you do better than that? Can you sort of intercept the thoughts before they have been packaged into something like concepts and language and so on? I'm somewhat pessimistic about that. I think that in. Well, one could say a few things. I mean, first of all, our brains kind of operate on millisecond timescales and electronics operates these days on nanosecond timescales. So there's a factor of a million difference. And yes, one could imagine that there's some piece of my brain where you can say, let's just take that piece of my brain and let's look at all the inputs that are coming in, all the neural, all the electrochemical processes that are going in there. And let's say, let's replace that part of the brain with something that does the same operations but a million times faster. Yes, I can imagine that being a possible thing. The interface between that and the essential me as I am, so to speak, is tricky. I'm not sure that, that, you know, in some sense, if you replace the whole brain with something that's a million times faster, well, its experience of the world is very different because at a million times current speed, the way we look around and see things and so on, we won't be seeing these integrated collections of sort of images. We'll be seeing individual photons arriving and all kinds of other complicated things. That is not part of our current experience because our brains are not operating at that speed, so to speak. And I think this question of to what extent kind of having the sort of one thing is the neural implant. That's the kind of, the kind of you already package it up in something that is like a Crispin thought and send it off to your computational system to figure out the answer, so to speak. Another is where you're replacing a piece of neural tissue with something which is plug compatible, does the same as the neural tissue did, but it's much faster. Those are two different things because in one case you're doing something that is completely alien to neural tissue, you're doing a computation. In the other case you're doing something which is plug compatible with what the neural tissue does, in which case you're really just thinking like a human thinks it might be a lot faster. And quite how one interfaces between those two is not clear to me at least. And I think that the essence of the thing that makes us be doing human like thinking, as opposed to the kind of thinking that sort of happens in the natural world. Let's say whether weather has a mind of its own or something, and it's doing all kinds of computations, but in a way that is very much not aligned with the way that we humans think about things. So I think this question of sort of, to what extent can we have something where we humans kind of care about it and think it's like human like thinking, while still connecting it into sort of very different technology. It's not clear how that will work. Let's see, it's a comment from is what extent can AI follow the speed of our mental images that sometimes we can't follow up with, not only in terms of Communicative language, but in terms of recognition. Well, so the, the question I think being asked there is, let's say we could generate images, our mental images, really, really fast. Could we extract information quickly from seeing somebody else's mental images? I don't fully know the answer to that. I mean, my experience in generating lots of unexpected images from the computational universe is I suppose that, yes, you can get things from those images pretty quickly, but the things you're getting are probably things that fit in with your sort of conceptual framework, which maybe you don't have words to describe, but it's still something where you are taking a thing that you're somewhat familiar with and fitting it in. In my life, like long, long ago, I came up with these four classes of cellular automaton behavior, which seem to have been useful in lots and lots of places. And that's kind of a case where I'd seen a zillion images. And it's like, how do I describe what's going on here? And I realized there are basically four classes of things that happen. And so kind of I just numbered these classes, and I could have given them exotic names, but I didn't. I just numbered them. And, you know, that becomes a thing that you can now think in terms of. Like, that's a class four like system or behavior. That's a class two like behavior. And to me, and to many people now, that immediately implies something. So it's a case where that was originally communicated essentially in images and then got sort of attenuated down to just being this kind of linguistic form. And quite how sort of, what is it that we can ultimately think in terms of? Can we only ultimately think in terms of human language? That's been certainly a philosophical theory in the past. Or can we, for example, think in terms of images? I think some people definitely can think in terms of images. I'm not particularly good at, like the 3D mental rotation type thing tasks and so on, but I know plenty of people who are, and I don't think they can, you know, if you say, explain that to me in words, you know, when you rotate the dodecahedron in this way and that way, you know what happens. And they can immediately say, well, then you'll see this triangle and this and that and the other. But the words are coming after the mental image came. They're not the mode of thinking there. And so I think the question is sort of, what is it that we can think in terms of? Do we have to. It could be the case that to think we have to turn everything into something that is basically like human language. I don't think that's the case, but I can't describe necessarily what. And as I'm talking to you here, my only means of communication other than these, rather, you know, rather than these weird gestures I'm making is my only real form of communication is the words I'm saying human language. So, you know, it, it, it waits for the future to have my sort of generative AI system show the mental images that I have alongside all of this. And then one can get a sense of what is separately communicated by that. I mean, I suppose in, you know, we already have some example of this in, in movies with sound and so on. We have, we have the words and the sound. We know in movies is pretty important to the way that people perceive the movie. More important sometimes to have the quality of the sound be good than the quality of the picture be good. But again, that's a case where we are sort of, we're using the image which was created in a movie, typically with great effort, not sort of on the fly to represent a new thought. We're using that as an important part of the communication of the message. Let's see, Elsie is asking, keeping with the future of art theme, will there even be a place for human artists in the future or will generative AI make it mostly obsolete, say decades from now? You know, before there was kind of desktop publishing, computer based artistic things, everything you made had to be made by hand. And all of those kind of decisions about how you lay out, you know, the letters in a, you know, I remember when I was a kid actually setting lead type and you'd have these actual letters and you would put leading, you know, pieces of lead between these different, between different lines to determine how far apart they should be. Those things were, were sort of deeply manual at that time. And then desktop publishing came in a bunch of, a bunch of aesthetics got automated. So you know, there's a, there's, there's a layer of graphic design, presentation and so on, which is in a sense computer assisted at this point. And it's kind of on some kind of rails that doesn't require every pixel to be human determined. And I think in all sorts of sort of forms of art, what we're seeing is just another stepping up of that kind of, you don't have to determine every pixel yourself. The same is true special effects in movies or another example of a place where this is happening. But does that mean that there doesn't have to be an artist underneath. No, it doesn't mean that because insofar as the purpose of art is to communicate from mind to mind, if there was no mind to begin with, that's not a sort of a form of communication that's going to work. I mean, it's an interesting thing when you generate forms just raw from the computational universe. What's happening there? It's not a human to human communication. It's more a nature to human communication, which is a different kind of thing. And it has its own effect, but it's not the same kind of thing as, I think, the typical role of art, which is human to human communication. And I think. So my feeling is that the expectations about sort of. Well, for example, for movies. Look back at the science fiction movies of the 1950s, for example, and the special effects there look incredibly hokey to us today. It's like, yes, that robot was really made of vacuum cleaner parts or something. And we can kind of tell that, you know, it may or may not affect the story that we can tell that that's a hokey, you know, hokey robot, like the original Lost in Space robot or something. But the, you know, whereas, you know, as. And then the expectations become higher and we expect that, yes, it'll be generative AI and, you know, there'll be. There was the moment at which people started being able to have kind of creatures, you know, the. The swarm of bats where every bat was kind of independently computationally determined. And then there were the, you know, the walking dinosaurs and the herds of dinosaurs. I happen to know that our Wolfram language technology stack was. Was used from the beginning, actually, in. In kind of figuring out the motions, the walking motions of dinosaurs and so on. And that. That code, I think, was sort of gradually used in many different places for sort of. If you. If you saw a dinosaur walking, it was solving equations with Mathematica and morphing language. But the thing that the expectations were raised, and I'm sure that will happen again, and it will happen with sort of typical artistic forms. You see. I mean, for example, one thing you already see today is when there's kind of a little iconic, sort of almost like in an illuminated manuscript, where the beginning of some section would have this very elaborately constructed first letter that had all kinds of pictures inside it. Well, now the icons that might start a section of a document, they might be nowadays generated by generative AI and they have a certain. They give a certain mood to things. And that is something that. Now there's an expectation that those things are fairly elaborate rather than just, oh, it's a block of red, another block of blue or something somewhere else. So, yes, I think there'll be a raising of expectations. I wouldn't say that the kind of the, oh, there's no human underneath. I don't think that will be a thing. Because the purpose of art is usually in the end, human to human communication. Except insofar as it's kind of like, oh, we've got this wall to cover. And it will be better if this wall kind of looks. Isn't just a plain white wall, but has some kind of texturing on it. And let's make that texturing interesting. It's kind of ambient art rather than art for sort of human communication. And I can see that more and more of that will probably be done in an algorithmic way. I mean, people have used a lot of things that I've made for that kind of background texturing. And again, why does that work? I think it works because we're used to seeing texturing and so on in the natural world. We're not used to seeing a big slab of white wall in the natural world. We're used to seeing some rock face or something that has lots of texture in it. So. And the way that texture is generated in nature is not that different from the way it's generated in these simple programs. And so we can make a sort of artificial nature form of texturing, which is sort of comfortable to us. Let's see. Sean is commenting again that art is an idea in the artist's brain that hits the friction of the medium. An instrument in music or paint or play. And visual art, AI art may become much more interesting once it has more actuators. Yeah, I agree. I think that when we can routinely. I mean, there's a bit of a challenge because we have to go from our minds to. Well, I think it is true that we form mental images which we cannot communicate. We imagine shapes that we can't readily implement. You know, to mold them out of clay is a complicated matter. You know, it's been surprising to me that in the world of VR and so on that we don't yet have the kind of moldable 3D objects, at least not well developed. It's hard to specify 3D objects with typical computer systems, typical CAD systems and so on, but yet we can form an image. And maybe, I don't know because I've never really tried to do it, because these systems haven't existed. Once I formed an image of a 3D thing could I gesture with my hands and easily kind of mold the thing. If I had a certain number of kind of rails for doing it. I mean, if it's purely molded out of clay, I've got all the mechanics of the actual operation of clay, and I can't move it that quickly, and it's kind of pieces fall off and all kinds of things like this. Whereas if I could imagine just gesturing in the air and having some VR experience or AR experience, where it's kind of like things are forming and there's, you know, and it's automatically symmetrical, or it's, I, you know, just touch something and it automatically turns green or whatever else, you know. That might very well be a way in which I can take my sort of inner conception of a visual form and in 3D and be able to actualize it. But I think the issue is that we only have the senses we have. So even if the AI had this magnificent swirling thing, well, let's even imagine that the AI could sculpt an arrangement of molecules in the air to make this complicated spatial olfactory experience. I don't think we do very well at understanding that. Dogs might do well at it, but we wouldn't do terribly well at it. And so even though the AI or the actuator is brilliantly spraying these molecules in this complicated arrangement, it's like we just maybe notice, oh, it smells funny here. But that's as far as we'd get. Unlike a dog, for example, that might, in its conception, whatever that may be, have this idea of the spatial olfactory landscape that's very elaborate. But I think we are sort of hitting against the sort of limitations of our own senses and our own kind of things that we're familiar with. I mean, imagine that we were connected to a million Iot devices, a million sensors. Instead of touching things with our fingers, imagine that plugged into our brain are the endless sensors, even cameras, for example, from around the world. But let's say sensors of other kinds of things from around the world. We would have kind of the nervous system of the planet, so to speak, connected to our nervous system. Would we be able to make sense of that? What would we do with that? Would it be the case? Would we, you know, beyond just sort of doing the data science on the fly of saying, oh, let's just filter this down to the, you know, is there a big explosion happening anywhere? Or something like this? You know, would we be able to really get a sense of the kind of planetary nervous system in the same way that we can? That we are familiar with getting a sense of what's in those 10 million pixels or whatever it is that we have in our visual field that we're used to because our brains have been evolved and built up and experienced at dealing with things like visual images, but we don't have that experience with a million IoT interfaces and so on. Okay, comments here from. Ryan asks, do you believe neural interfacing can increase observer capacity? Yes, but with the comments that I've made, namely that in the end, sort of, it's one thing to say my AI knew what was happening. It's another thing to say I knew what was happening. We have our own inner experience, and that's what we probably care the most about. To say, yes, I've got this AI control system over here and it was very aware of what was going on. It just never communicated that to me. It doesn't make it. There is the, you know, me, I care about me as an observer, so to speak. My thread of experience, the threat of experience of an AI over here that's doing all kinds of different things is a different matter. And interfacing how tightly I can interface the sort of experience of the AI with my own experience, that's not yet very clear. Sam comments. The idea that brains operate on millisecond timescales seems wrong. Brains are not digitized control loops. They're continuous systems. Well, actually, they're a lot less continuous than you might think. I mean, if you look at the spike train, so a neuron is, you know, an electrochemical system and it is sending out these electrical signals. And if you, you know, connect it to a loudspeaker or something, you'll hear these, you know, you know, you'll hear these, these spikes of activity from the neuron. The thing is, is discreetly spiking and the rate of those spikes is on the order of, of milliseconds. That's the maximum rate of the spikes. The, the change from its spiking to its not is slower than that. And even when it comes to the connections with the synapses which connect the dendrites from one nerve cell into the axons of another nerve cell, the kind of the output going to the input to another nerve cell, what is actually going between those things, what's being communicated is little vesicles of neurotransmitter, which again are discrete. And when you have know, I think a typical firing of a nerve might have a hundred vesicles, but maybe less, but, you know, eventually you get down. It's Kind of like with. With microprocessors today, every bit is like 100,000 electrons. But you can imagine getting all the way down to a few electrons per bit. I don't know. At brains, I think you can get up to. You can sort of saturate if you're. If you're pushing neurons too hard, you'll kind of saturate the number of pieces of neurotransmitters. So even at that level, there's more discreetness to brains than one might have expected. Now, in terms of, is it the case with digital electronics? As microprocessors are built today, there's usually a central clock that says, okay, fetch the next piece of memory, do the next instruction. It's a synchronous clock that is sort of telling the whole microprocessor. It's kind of flowing across the whole microprocessor saying, do this next, do this next, do this next. We don't really know how that works with brains. The fact that there's eeg, that is sort of a global rhythm that operates around 10 cycles per second, something like this, give or take, is interesting. It's not completely clear, I think, what the role of that in potentially providing synchronization across the brain is. One knows that if one messes up that. That frequency by driving the eyes with something close to that frequency and kind of attracting the nonlinear oscillator to a different frequency, you can lead to all kinds of terrible global problems of global electrical effects that lead to things like epilepsy and so on. So there's some kind of clocking mechanism in the brain. It's not completely clear how it works. Some global clocking mechanism can, just as there is in a microprocessor. Although the timescales for the EEG clocking mechanism are incredibly different, on the order of maybe tenth or a hundredth of a second, whereas the ones in a microprocessor are more like a billionth of a second. But I mean, this question of whether there is something that is sort of a digital lockstep operation that's happening in the brain. No, it's mushier than that. Does that matter or is that just an implementation detail? Microprocessors, because of the way they're engineered, are designed to operate in this kind of lockstep, synchronous way. It's not clear that it has to be that way. It's just that if you want to engineer it and be able to foresee what it will do, that's a way to be able to foresee what it will do. I think even things that have come out of our physics project suggest sort of approaches to distributed computing that are much less. Every bit has to sort of be updated at the same time and perhaps a bit more like what happens in brains. But I think one of the big lessons of modern AI and LLMs and so on is that things that we thought we just had no idea how they worked and we had no models for, we actually have pretty good models that are, you know, sort of computational neural networks that really operate in many ways much like sort of microprocessors, although they have detailed differences more than they operate like the kind of biological, electrochemically understood brains. I mean, one difference, by the way, is that in typical. Well, in digital computers, everything is represented as bits, discrete bits. In modern neural nets, actually neural nets, ever since they were invented in 1943, the key elements are numbers, real numbers, 3.7482, not just 3 or 4 or whatever. And that's sort of a difference between neural nets and digital computers right now is that neural nets have these continuous numbers. Digital computers have discrete bits, bit based numbers. And that has been important for neural nets because when you train a neural net, you're typically tweaking it one tiny infinitesimal step at a time to get it closer and closer to the thing that you want it to learn. Actually, I don't think that's essential. And in fact, things I've just been working on myself very recently kind of suggest that there are ways of thinking about machine learning that do not involve continuous numbers like that. And that's been sort of a feature of the way that neural nets have been thought about, because neural nets were invented before digital computers were invented. And they were invented at a time when sort of a dominant theme in understanding how things work was calculus based mathematics, which is what neural nets use to make use of when they do things like training. Elsie is commenting. Could neural technologies, together with near speed of light transfer speeds between people make the latency. I'm not sure, referring to what latency become negligible. You know, we have remarkably little latency today. I mean, the fact that, you know, I'm sure people are watching this, my, my efforts here from all over this planet and you know, with essentially no, no, no humanly perceptible latency. So it's, you know, we've already achieved remarkably low latency in, in pretty global communications. I mean, it'll be interesting when, you know, how social norms will adapt if there's A Mars colony one day when there's, you know, 20 minute latency, necessarily 20 minute latency. I mean, the fact is people adapt to things very quickly. I mean, back in the day there was, you know, it was, you know, hand carried letters and carrier pigeons and so on. And you know, society did manage to operate perfectly well then. Today there are plenty of low latency activities and other ones, like email, for example, where the typical practice is a higher latency response rate. Spare Parts is commenting. There are some great 3D sculpting tools in VR. I need to see some. Let's see, Let's see. Spare Parts is commenting. Apparently there's a vast difference in different people's ability to visualize images in their minds. Interestingly, they say many artists seem to lack this ability. That's interesting. I didn't know that. Yes, you know, the thing to realize is that different minds do operate in different ways. Does it in the end matter? If you want to be a high jumper or something, you better have certain muscles and tendons and whatever else and length of legs and all that kind of thing to be able to do that. There are presumably similar aspects of brains, although we understand those much less well, where if you want to be a sort of 3D visualizer of things, you better have that well developed kind of piece of the brain, like the long legs or something that relates to that. I mean, the thing that is interesting about brains and confusing about these kinds of things is brains have a certain universality to them. Just like computers can be universal computers in the sense that you've got a fixed piece of hardware, you feed it different software and it does different things. So similarly, if you say to a person, a brain, do this thing out there in the world, there may be more than one way to do that thing, there typically will be more than one way to do that thing. And different hardware brains may still be universal enough that even though the brains are different, they can still achieve the same things. I mean, I always notice that with, with people doing things like running companies, there are very different kinds of people who run companies. Yet if the objective is run a company, you can do it, even though probably the raw sort of hardware ability of brains of those brains is different. Now there are some things where you can push out into different corners of human capability, like 3D visualization, like various kinds of rapid math or, or games problem solving. These kinds of things where you're kind of poking out into corners of the capability space where you can't sort of paper it over with kind of other forms of thinking, so to speak. I mean, if the problem is visualize this 3D thing, probably for somebody like me, who isn't terribly good at that, I could think to myself, well, gosh, let's say I can say, well, let me think about group theory and let me try and do a mathematical analysis of this and let me do this and that and the other. And quite possibly by that route I might be better than the in brain visualizer, so to speak, by going through that different route. I will say that back in the day when I was a kid doing physics and so on, and people were doing all this kind of mathematical calculation and working out integrals and all this kind of thing, and they were very proud of all these tricks they'd figured out to work out all these different kinds of integrals and so on, you know, I ended up kind of. Well, I used computers to do these things, but I also, from using computers I developed an intuition that allowed me to have some very different ways to do things like integrals. And in fact, in the end my ways of doing them ended up looking very impressive and alien to people who were doing them the usual kind of way. But what was happening was that I was kind of. I was not having to exercise the part of my brain that other people were doing really well with all those tricks and collecting together all those pieces that fitting the puzzle pieces together. Rather, I kind of knew some more abstract things that were kind of sort of industrial machine type methods for doing integrals and so on, which I had kind of learned from watching computers do them that I could then apply through a different path, but getting to the same result. And I think that's a very typical thing to see with brains. It's sort of different from the more mechanical kinds of things that people might or might not be able to do. It's something where, when it comes to thinking when the end result, when what you're trying to achieve is a particular end result, there are many paths to get there. Actually. It's sort of interesting that the things I'm looking at right now with machine learning, where I'm able to kind of see some of the foundations of machine learning, that phenomenon of there are many very different ways to think about something to come to a conclusion. You can see that very directly in the things that I've been looking at. And by the way, that also relates to kind of brain to brain communication. Because if there are things which might lead to the same words being said, but the actual internal construction of the thoughts is Very, very different. That kind of doesn't bode well for the attempt to transfer the underlying thoughts rather than just the kind of, the final sort of crispened up linguistic form or whatever else. All right, I think I have to go back to my day job here. Fun though this is. And thank you all very much for lots of interesting questions and comments here. And. I see lots of. Oh, I'm, I'm. I'm led to one particular one from Javier, commenting on the discussion that I had. When was it? Last week, I think, with a robot, where the robot said it liked to tell jokes and make people laugh. How is it possible, Javier asks, for robots to develop their own personalities outside what they are programmed to do? Nobody really knows how to program the personality of a robot. The best we know is to give prompts and to use the fact that if you tell an LLM, talk like a pirate, probably not a real pirate, but a conceptual pirate or a stereotypical storybook pirate, talk like a storybook pirate, that it will know how to continue from that, remembering what it's read essentially in storybooks and continue that way. Now, is that fundamentally different from how humans develop personalities? I'm not sure. I think that, again, it relates to what's sort of in the hardware of the brain versus what's kind of something that is a particular way of thinking, let's say, that develops from observing other things and experience. I mean, I think the fact that. I think it's unsurprising that LLMs, for example, can end up developing personalities. And I think it also probably is the case that when an LLM starts going off talking like a pirate and then it hears itself effectively talking like a pirate, that sort of reinforces its pirateness, and it's going to go on in that way and it's going to build a whole sort of, in a sense, I, I was going to say worldview, but it isn't really. I'm not sure we can, we can attribute it. We can, we can say it. It's. It's like a worldview. It's effectively building a worldview that comes from its experience of talking like a pirate, let's say. And I think that this idea that, you know, you can end up with sort of very divergent personalities. In fact, somebody at our summer school this year did a study of AI personalities. You can find it on the web, where it was looking at kind of when you prompted it in different ways, how did that lead to different descriptors for personalities? And what did it respond on these personality survey type tests. So I think. I'm not sure that robots develop personalities in any really significantly different way than humans do. It's sort of a mixture of the experiences they have and the particular details of the neural net as it was constructed, as it learns and so on, just like that. Probably aspects of the way we are that are the result of particular specific details of how our brains are constructed. But it's hard to tease those things apart. And it's, it's, you know, and as I say, there's an awful lot of kind of universality that papers over those differences of hardware. All right, time for me to go back to my day job. So thanks very much and see you another time. Bye for now.