Identity Play and the Artist with Holly Herndon

Holly, could you just give us a, a brief description or resume of, of where you are at, what you are doing at the minute and, uh, like the kind of field you operate in, or maybe, maybe that's not easy to define.

Yeah, I'll, I'll give it a shot. Um, so I'm an artist. I, uh, I have a background in computer music. Um, I, that's what I studied. Um, and I've been working with, uh, AI for about 10 years now. And more recently I've kind of, uh, made my foray into the, um, to the art world with a solo exhibition at the Serpentine in London. And my exploration of AI for the past 10 years has taken me down some pretty psychedelic journeys, one of which being exploring my identity in AI models and this idea of being able to create models based on my own IP. It's an idea I call identity play where other people can kind of perform through me. It's a project called Holly+, who you also welcomed to the program here. So I would say that I'm an artist working with technology.

Yes, because you studied that, did you? You studied technology, am I right in saying?

Yeah, technically I studied computer music. There's this really special program at Stanford where I did my PhD, which is— it's a very technical program, but it's also housed within the music department. I really feel like I'm a kind of translator between technical fields and cultural fields, and that's where I feel the most comfortable.

Maybe that is also more akin to the future. We might get onto that towards the end of the conversation. But I first discovered you through Holly+. And it wasn't really your TED Talk or anything. It was just— I think it was a few years ago when I was researching the book. And I couldn't find that many people who were exploring identity. But there were a couple of people in the sort of art, techno, music field. How did you come about to conceive of Holly+?

Well, as I mentioned, I've been working with AI for about 10 years. And this is really before it became really productized. Ties. So when most people think of AI today, they think of ChatGPT or DALL·E or Midjourney. These are really large general models that are trained by scraping data from the internet. But back in the day, none of those products were available. So you really had to train your own models. And one of the first questions would be, what do you want to train the model on? What's the dataset? And, you know, with a background in composition and understanding the music history of kind of sampling, I didn't feel comfortable using anyone else's data. So I just wanted to use my own data, put together an ensemble where we recorded training data specifically for training a model. And when I started working with specific voice models, I felt the most comfortable training on my own voice. So a voice model is essentially an AI model that learns all of the sounds of your particular voice that can then emulate and sing in your natural singing voice. And so I felt weird asking someone else to do that. It made the most sense to kind of experiment on my own voice. And I am a vocalist, you know, I've released records where I'm singing. I never really thought of myself as purely a vocalist. I feel like there are many, many people with much more beautiful voice than I have, but this felt like the kind of, I don't know, the ethical approach to use my own IP in that instance.

Yes. And how important is the emerging sort of field of decentralized technologies, whether it's AI or blockchain or the new era, if you like, of less sort of linear hierarchical technologies and the new sort of networked technologies. Has that made much more of your work possible?

Yeah, I think this has been a really fundamental part of what I've been working on. I mean, one way that we like to think about it— so I released an album in 2015 called Platform, and then an album in 2019 called Proto, short for protocols. And I see this kind of shift from the platform era to the protocol era. The platform being the, you know, you think of like Spotify or Instagram or these specific products that are platforms that kind of determine how artistic expression is distributed. And so different decisions that the platform owners will make have a direct impact on how media is made to optimize for that platform. I like to think about us entering this kind of protocol era where we're working at a layer underneath. And if you think about protocols being, you know, something like Ethereum would be a protocol that you can actually build applications on top of. But you can also think of a protocol as like just a way of behaving, like certain kind of like social manners that we have are also just kind of like social protocols. It's a kind of rule set or an understanding of how things work. So I like to think about the creation and distribution and all of the kind of facets that go into the art-making process as part of the protocol, as part of the artwork itself. And yeah, I'm hoping that we're entering an era where more artists are able to have more control over all of those facets of their output.

Yeah. Do you think many do at the moment? I mean, there are some people working in this space, but not many who are as high profile and prolific as you. Do you think that's going to change over time now?

Yeah. I mean, I think a lot of the work that I've done requires really boring research. And I think a lot of people, you know, fair enough. People are busy and they don't want to have to read machine learning papers and watch academic conferences on YouTube. That's a pretty specific thing. But yeah, I mean, I think there's a technical barrier for entry for a lot of these things. But I think as— I feel like we learned so much from the internet becoming a really ubiquitous, all-present part of our daily lives. And I think people started to wake up as to how technical decisions or platform decisions, design decisions, were impacting their personal lives really directly, I think people have a desire to have more agencies within these systems. So yeah, I do think people are becoming more and more interested in understanding the tech and understanding how they can have agency with it.

That's interesting because some of your work in the past has focused on the theme of surveillance, hasn't it? You know, how we are being monitored or surveilled, privacy, public. What does it mean, do you think, to be machine readable, to be noticed, to be captured, monitored, analyzed, maybe even redefined or represented or represented in this kind of emerging tech world?

Yeah, that's a great question. I actually have a book coming out in the next couple of weeks called All Media Is Training Data. And that doesn't— some people read that and they think I'm saying all it is is training data. And that's not what I'm saying. I'm saying that, yeah, as soon as something is captured in media, like, you know, this conversation that we're having, or as soon as you take a photograph or a video of anything, then it becomes machine legible, and then it can become part of a training canon, you know, to train a model that you may never even interact with. So a lot of the data that we're placing in social media, online, you know, this is already going to train different models that we're just not seeing or necessarily interacting with. I think it's a pretty profound moment in our kind of on our human timeline.

Yeah, I would completely agree. I mean, I think people are kind of used to like, oh, maybe something's extracted from a PDF about me, me or a Wikipedia page. But all this kind of digital phenotyping, treating us almost like patients, capturing some of the atmospheric or environmental data around us, the way we move. I was talking to somebody about motion prints in VR, for example, the way in which our gesture with our head and our hands, it's so individual that it's like a fingerprint and they call it motion print. I feel like sometimes there's nowhere to escape. Maybe that's the dystopian view of it. And obviously you are looking at it primarily through the opportunities, but It is a very, as you say, very profound shift, isn't it?

Yeah, I think it all depends on kind of where we end up in terms of, you know, who's using this data and for what purpose. You know, on the kind of dystopian side, okay, you just have giant corporations that are just, you know, scraping up all of our data and they're monetizing it in ways that we don't participate in. And that's the kind of like dark image. But there's also, I think, a beautiful way to approach AI that requires a lot of work being put in the public domain, having public domain models that we can then fine-tune tune and build on top of, participating and contributing to something that feels like it's collective, like a collective accomplishment. I like to think of AI as, as us in aggregate, as a collective human accomplishment rather than a kind of like alien other that's here to just replace us. So I think if we can think about it as a kind of really sophisticated coordination and collaboration mechanism and we can find ways to, to make it work for everyone and be fair, then I think it maybe will be less scary for people.

I like that idea of complicity. That's a balance between the human and the machine elements. I think you've done that really well with Holly+ because obviously you've invited people to sort of be the, I don't know, the stewards of your voice in the way that they deliver it, haven't you? I mean, and I think that's, I don't know, what was the reception to that when you suggested that? Because I saw a lot of people online saying, oh, but what does this mean for sort of rights, copyright, royalties, you know? And actually, I think, and correct me if you're wrong, but you I'm saying you can use my voice as long as you credit it or give me attribution, but what you do with it is up to you artistically, maybe to a point.

Exactly. And that's what works for me. That's not going to work for everyone. And that kind of goes back to the conversation we were having earlier about the platform dynamics and this kind of one-size-fits-all for artists. I hope that we move to a situation where people are able to really make specific decisions about their own practice, the economics that make sense for them and their communities. For me, I was fine with just taking a permissive approach my IP. I was more curious to see how people would use it. But I could imagine another vocalist might really not want someone to use their voice. And that's totally understandable. And that right should be there. At the moment, it's a really— it's a gray area. You have personality rights as something that has more legal history. So if somebody recorded something and then used my name as an artist name, you could sue on those grounds. But it's less difficult to sue on the training of a model using my voice. So we'll see where all of this lands. In terms of legal. But yeah, that's just what felt right to me. I wanted to just experiment and make it as open as possible.

Did anybody do anything with your voice that you weren't happy with, or maybe surprised you, or was innovative in a way that you hadn't really expected?

You know, no one did anything that I was offended by. You know, also, the voice that we made public was a— I have multiple voice models. One of them is more kind of language-specific. Another one is a real-time instrument that people can sing through. And then the one that we made public is a more kind it deals in more kind of like ambience and texture. So it can take a polyphonic input, any kind of sample you can drag onto the website, and it'll transfer it into a kind of choir of hollies. And because it's more abstract, I wasn't really dealing with having to edit out hate speech or anything like that. Of course, that's always a question when you're dealing with the open internet. But really, I was surprised at the kind of beautiful approaches that people took to it. I mean, one person recorded their train ride and the kind of screeching of the train, of the train tracks, tracks to the metal and then transferred that into my voice and it made this gorgeous choir with the most amazing overtones and just things like that that I wouldn't have thought of myself were just a really nice surprise.

You see, I think this is one of the really interesting dimensions of AI, that it gives you another perspective once you apply these tools or use these tools. And I think Eric Schmidt, of all people, said this about AI agents that, I don't know if you agree, but that they will give us another perspective to blend with the perspectives that are delivered through our own human senses and give us a picture or an understanding or even a window into the world that is richer, we might not have had access to before?

Yeah, I mean, AI models essentially are just able to consume and crunch and make sense of a much broader or a vaster, if I can say that, amount of data than the human brain can. And so, rather than this kind of replacement thing, I see it as an aid to help us help us understand the vast amounts of data that we're able to collect these days. So yeah, I think that's true. It's also just kind of like a strange mirror to ourselves, a strange reflection. It's not exactly us, it's a kind of mutant form of us, and so by interacting with that mirror world, we understand ourselves a little bit more.

It's weird, isn't it? Because it's got an intimacy but also an alien nature to it. It is that weird— maybe that's the uncanny thing about it.

Yeah, I think the, I think the intimacy is that it is actually human created. It's all of the data is from humans. And so it's basically you're just interacting, you're collaborating with vast amounts of people and our brains haven't really evolved to be able to conceive of, of that. You know, we come from tribal groups where we're used to having only so many kind of social connections and now all of a sudden we're able to collaborate with with millions and millions of people at once. And that, I think that that scrambles our brains a little bit.

Which is one of the reasons I love the language that you use for these things, like spawning. I love, 'cause it's biological, isn't it? Not technological.

Yeah, so Spawn, it started out as Spawn when I made Proto and we were working with our ensemble in Berlin. We wanted to have an ensemble member trained on our voices. And so we came up with the name Spawn and called her our AI baby. And spawn has since kind of grown up into a verb. This word spawning is what we like to use for some of these generative AI tools. So instead of sampling, which is a kind of one-to-one mechanical reproduction, we think about spawning as being able to create infinite media from a kind of seed source. And so spawning felt like a much more kind of descriptive version of what's happening there. And then spawning has evolved once again and is now the name of an organization that I co-founded with my partner Matt Dryhurst and Jordan Meyer, who's a Machine Learning Engineer, where we're working to advocate for data manners in the machine learning space. So Spawn, spawning, spawning has had quite an evolution over the last several years.

I love it.

Yeah.

So is that right? So it allows artists to opt in and out of datasets. How does that work? How does an artist access that?

Sure. So at the moment, it's in the image space and the text space. And so you're able to go in and at the moment, we don't have opt-in, but we do have opt-out. So if you don't want for your images on your website, for example, to be included in machine learning training. You can add that to your website using our tools. You can see if your data is already being used in training sets by going to haveibentrained.com. Yeah, just check out Spawning. We have several different things in the works. We're also working on the public domain. At the moment, we see that as a really important first step for an opt-in. So if you have a public domain base layer model, model, then you can fine-tune that model on your own work and feel like you as an artist have actual ownership over that model. So you can then sell the model, or you can license the model, or you can come up with whatever economic interaction that you want around that model, knowing that you haven't infringed on anyone else's copyright. We see that as a really important first step to getting a lot of artists into the space.

That's amazing. So have you got a big tech team then, or is it just you and Matt and a few collaborators doing this?

It is definitely not just me and Matt. We have whole team of wonderful people. You know, we've always been collaborative-oriented. Can I say that? We've always been collaborators. And I've, I've always really enjoyed working with teams of engineers. So Spawning is its own kind of thing. It's run in the United States and there's a whole team of people working on that full-time. The work that we did at the Serpentine, there are 3 different novel models that we're working with for that show. And we're working with IRCAM, which is a research institute in Paris. We're working with Algamos, which is a research team in Lille, Stable Audio, which is a decentralized kind of audio research organization. So we have collaborators and engineers that we're in constant contact with that really make everything possible.

That's amazing. That really is a network, isn't it? A kind of vibrant, dynamic network. Let's talk about your work at the Serpentine. I think it's on till February, isn't it? Mm-hmm. Because I really want— yes, I'm hoping to come and see it before it It finishes in February. I know it's live now. Could you tell us a little bit about it? Because it's quite the construction.

It is. It was a— Yeah, it's quite the project.

Beautiful as well.

Thank you. It took over a year to put together. You know, we worked really closely with the arts and technology team there. Hans Ulrich is so amazing at understanding what's happening and staying on top of all of the kind of cutting-edge developments. And so he really trusted us with the vision to work with the arts and technology team there. And what we wanted to do is— so many shows really focus on just the output of a model because it's so impressive. What we wanted to do was focus on the process of the making of a model because we see making models as an artwork in and of themselves. Every step in the process, there's a moment for artistic intervention. So the creation of really deliberate training data as artworks, that going into training with choirs across the UK, those Those are performances, those are artworks. The decisions that we make in categorizing the data for the model, the model itself, and then interacting with the model as an output, each of those phases we really wanted to show and give the visitor an experience of in the space. So we put together this really wild production. We wanted to make a choral AI model that could be collectively owned by a group of choirs. So we recorded 15 choirs across the UK. We hopped in a car and we drove around and we recorded all of these choirs. And so when you go into the exhibition in one of the rooms, you can hear the kind of field recording and the ethnographic recordings that we were making on site. So you can hear how the training of a model, it's not just these kind of random data points, it's actually people being creative and making meaning and making work together in a space. That's what these data points are. They're not these cold little little bits of media. So you can experience that. And then we created an organ out of GPU fans. So the sound of our studio is just the constant whirring of GPU fans that's cooling our GPU as it's crunching all the numbers. So you kind of can experience the compression phase and the power that goes into training these models through the organ. And you can hear the output of the model that we created with all of the recordings across the UK and our own personal archive. We made a model that we prompt and create fully coherent, beautiful choral songs, infinite choral songs. You can hear those spatialized in the space. And then also the visitor can interact with the model. That's a really important piece of this, that these models, they're designed to be interacted with. So we wanted to allow the public to see what that feels like without introducing a screen, because there's so much screen work when it comes to people interacting with AI. So we wanted to eliminate the screen and just use microphones so people can sing in through a microphone and they can kind of navigate through the choral model and hear the choral model singing back to them. So we hope that the visitor gets the experience of the kind of full-stack training of a model and maybe can see themselves in that and how it might be beautiful to collaboratively train a model, you know, with other people.

It really is amazing. It is that idea of the process being creative and also collaborative and transparent to people. A bit like your idea about protocols being habits and ethics.

It is! 100%. Yeah, I mean, I feel like the design of the exhibition, we were thinking very closely, what is the protocol? What's the approach? What's the kind of rules that we want to put together for how this functions in a space, for how we deal with the individual participants? That was a really important part of the kind of protocol design of the show. And one piece of it, it might sound really bureaucratic and kind of boring, but we worked really closely with the Serpentine. Their team is really at the forefront of thinking through, from a legal perspective, how you can co-own and collectively kind of shepherd a dataset. So they put a lot of energy into legal work and legal frameworks to think through how can we invite the people who participated in these recordings to play a part in stewarding this data moving forward. So yeah, Yeah.

What did they think of it? What were the participants?

You know, it was such a wonderful wide range of people. A lot of people were really surprised at the variety of ages that were involved because this was a, I think a lot of, a lot of projects would just record in London and we did do some recordings in London, but we really hit the road and went across the UK and we went to rural communities and we met with older people and people who had never heard of AI and we tried to get a really kind broad cross-section of experience. And people had really interesting thoughts and concerns. And in a way, it was informative to us to understand, you know, kind of where people are at. But I think we also were helping people think through their own data footprint online. Like some people, for example, would come to the project and be a little bit wary of training an AI, but that same person might upload their music to YouTube, not understanding that that is already training an AI. Just without their knowledge. So there was a lot of information that we were able to share with people to better understand how this landscape is unfolding.

It's interesting, isn't it? Because I think it's been really difficult to get people to engage with the idea or the notion that identity is transformed by the media that you use. And I guess that's a great example of that actually happening, although you might not necessarily use the language identity or personal identity. Maybe people start to realize that is happening to them, like elements of the self are extended or augmented or whatever. I think that's happened with avatars as well. I think now people are messing about and playing with avatars, people can start to understand, "Oh, hang on a minute, this can happen to my voice," or "Suddenly I can speak Japanese in my own accent even though I can't," sort of thing. Do you think it's going to get to a point where there is a better understanding of all this?

Yeah, I mean, that's something that I think about a lot, that I'm actively working to help people understand. One of the things that I think about is the kind of difference between the data point and the actual person, like the fleshy, messy person that's in the real world. So one of the projects that I worked on this past year was called "xHairyMutantx." And so that's a project that we exhibited at the Whitney Biennial this year. So this is a really funny kind of superhero character that I created for myself because I, my name Holly Hernan, me as an artist, I meet the threshold of being a concept in some of these large models. So if you go to an image model and you type in my name, a kind of essence of me will appear. And because these models are trained by just scraping the open internet, it's just collected a bunch of images of me and created a kind of pastiche of what it understands of me. So because I'm not as famous as someone like Dolly Parton. Dolly Parton would have all of her full features and would be much more fleshed out. Me, I just kind of appear as an orange blob. So, you know, where— wherever my face might always be different, but I'll always have this giant orange blob of hair. So I was really asking myself, okay, how much agency do I have in this, in determining who I am in these systems? If this is going to be the net— if this is going to be the next layer of the internet, do I get to decide who I am here? And so when I did some experiments and tried and introducing some images of me that were outside of that with entirely different hair, it was kind of thrown out as noise. So I realized if I wanted to have any agency here, I had to almost lean into my cliché, lean into my pastiche. So we designed this wild costume, X-Hairy Mutant X, which is this kind of Gollum superhero with giant orange hair. And then I created a model with my partner Matt Dryhurst of that character that people can actually go to the Whitney website and prompt through and no matter what you prompt, you'll always come up with something kind of XHairyMutantX-like in the image. It's kind of infected with my giant orange hair. And each of these images are tagged with my name. So all of these images then are online and the next time a large image model scrapes the internet and tries to understand me as a concept, it will encounter all of these giant XHairyMutantX images of me and that will transform who I am in the model. So it's a kind of long-form performance between me and the public and machine learning companies. And it's a really fun way to kind of explore identity and who I am and the kind of difference between the person and the data points and the digital footprint that we leave on the internet.

Oh, I love that because this is the area I'm really interested in. And it reminds me a little bit of Database Animals, Azuma, about the otaku and the otaku's obsession with anime, manga, and obviously— and I really see Let me know if you agree or not, but this sort of breakdown of us as people into this database of "kare mo," mo elements, like the attributes, the elements that make us up. And I mean, you just said, you know, orange hair, for example. And they're very much like these characters having loved elements, "kare mo." And how do these elements get mixed, remixed, arranged, and rearranged together, and then spun out in different contexts? And the idea is that In this fan world, there is no original, there are no copies. Like, the parodies are just as important as the original character. But these elements are kind of constructed and reconstructed and rearranged in lots of different ways. And that's maybe how we should think about ourselves in the digital environment. I mean, I know it's a kind of atomistic sort of way in these individual attributes, but is that a way forward for us to think about the self? Oof.

Yeah, I think that's a really interesting reference. I think it kind of intersects with this idea of fandoms and different characters that allow themselves to be kind of remixed and reinterpreted. Machine learning models are designed to be interacted with. They're designed to be prompted. And so this kind of idea of authorship just being a one-way stream or just being one person, you know, in order to make the model, you're usually drawing from many, many people's creative input in the first place. And then when you're inviting people to interact with the model, Then you have this kind of like fandom moment where people, the new work is, it's both you and it's the person who prompted and it's this interesting blending. And I think, yeah, I think that we can learn a lot from how fandoms have proliferated in the last several years.

Yeah, no, totally. You mentioned collaboration and I know obviously you've spoken a few times about the people and groups that you work with. What about your podcast Interdependence?

Yes, yes. Oh gosh, Interdependence. I miss Interdependence, but we just kind of ran out out of time. It was such a big part of our COVID time.

Will you carry on doing it? Because you had some fascinating guests.

I really enjoyed it. I would love to be able to— Yeah, maybe Holly Plus can take it on for me someday.

That's the point of our avatars, isn't it? They can go off and do things, or our agents, that we haven't got time to do.

Exactly. You know, I'll never say never. At the moment, we don't have time to start it again, but it's something that I very much miss. It was one of those things during COVID where we were so used to being touring musicians and traveling around so much, we missed you know, meeting new people all the time. And it was one of those socially acceptable ways to call someone that you find interesting and just talk for a while.

Yeah, that's true. That's why I got in touch with you.

Yeah, exactly. Yeah, so it's honestly, it's a great, it was a great social outlet for us and I definitely miss it.

But is there anybody on there that you thought, or an idea or a concept that was just so out there, challenging or amazing and inspiring?

Oh gosh, I don't know if I would single out. There were just so many good ones. You know, one person that for some reason is popping in my mind right now is Jacob Horne from Zora. He's been thinking about these kind of decentralized identities and co-ownership ideas and IP and new ways to approach that for a really long time. And I always— he comes from a different, very different perspective than I do. And I always feel like it's a fruitful exchange with him.

So Holly, I know I've got to let you go in a minute, but what do you think the future of identity holds? If we were to look sort of 20 or 30 years ahead or like mid-century, where do you think we'll be in terms of understanding ourselves, representing ourselves, expressing ourselves as artists like yourself?

I think this is a really interesting philosophical question. I almost wish my partner, Matt Dryhurst, would be able to answer it because he's been thinking a lot about how the concept of identity and individuality has really shifted throughout the ages of humanity. There's this interesting book that he's been reading called The Creation of the Individual. So I think one thing that might change is how we perceive ourselves as individual identities and might perceive ourselves more as kind of networks and relationships with other people, and also just our own biology, how we might understand that depending on the, the way that you look at someone, if like with the naked eye it looks like a single person, but if you look through a microscope, there's all these different interconnected desires and microorganisms that are kind of interconnectedly working together. So in a way, we as, as selves are also a kind of web of, of collaborators making our whole bodies work. I think that identity is going to get really weird And I very much welcome that.

Yeah, it's exciting, but it's also threatening because if we have 8, 9 billion people on the planet who are all in on identity crisis, because as you say, if it does get a bit weird and we can't understand the interior of ourself or selves as identity continues to be thought of in a more pluralistic way, let's say, I wonder what will happen to people, whether they'll be able to cope with Yeah, you know, I think, not to be like super wide-eyed and overly optimistic, but I think that's one of the roles of art, honestly, to help people understand or to help people work through the feelings of alienation in this like super technologically mediated world that we live in.

And I hope that we can contribute to that in a meaningful way.

Oh, I think you do wonderful work there. Thank you. It's been fascinating to speak to you. And I'm sure it's only the tip of the iceberg, what's to come. From Holly Co. So thank you, thank you, Holly. It's been great to speak to you.

Thanks so much, Tracey.

Thank you for listening to The Future Review hosted by me, Tracey Follows. Be sure to check out the show notes for more info about the topics we covered today. If you enjoyed this episode, please like and subscribe wherever you get your podcasts. And if you know someone who would love this episode, please share it with them. For more on the future of identity in a digital world, visit futuremade.group/thefutureofyou. To explore the future of everything else, head over to futuremade.group. The Future of You podcast is produced by Big Tent Media.