Cannes Round-Up: Mario Klingemann: Collaborating with Neural Networks

It's Machine Learning from ASSIST. It's a podcast where we think and dream about the future of AI, the talking internet, and how we're reshaping our culture. This chapter is an excerpt from our interviews at the 2018 Cannes Lions International Festival of Creativity. We went to Cannes to tap the brains and imaginations of an incredible gathering of leaders and innovators in technology, advertising, and brand communication. From our spot on the floating home of LivePerson, we were challenged, inspired, and learned a lot. Once again, all of us at Machine Yearning and Assist owe Rob Lucassio and his team at LivePerson a huge thanks. LivePerson took great care of us and are a true friend of the pod. Now we have visual artist Mario Klingemann. Mario's been taking neural networks and turning them into his artist's toolkit. You absolutely must go check out his work at quasimondo.com. Mario Klingemann, enjoy.

Everyone says the word neural network, but I don't believe anyone explains it well. And you're very, you're like an expert machine learning person. And so can you please explain it to me as if like I was just turning 6 years old in kindergarten and I was like, mom, I love neural networks. Like, but she's like, what is a neural network? What is a neural network?

I'm not a computer scientist. Okay. But the way I understand it, so actually, let me give you a metaphor.

Please.

Imagine a laundry machine, like the one where you put in all your jeans and socks and everything, and then you wash it and then you pull it out. And that is your data, actually. And maybe, well, I don't do the laundry that often, but usually things are kind of intertwined and twisted and something, but Imagine every piece that is in there is data and the jeans is the dogs and whatever you pull over is the cats and the socks are, I don't know, mobile phones and now the machine tries to tell them apart and what the neural network does is kind of find the best way to untwist this ball of laundry. So in the end, it lays them all out nicely in a certain plane, and then it can say just, okay, on the left is, is my jeans, then next to it is the pullover. So, but in the end, it's like you untwisting the laundry by a mathematical operation. And so, but the data is kind of hanging together. The data is the leg of your jeans, which should stay together. You don't want to rip it apart. And so, yeah, it's kind of a weird rotation transformation. That's all that happens inside of these neural networks in a way, a way to untwist or twist the data. So in the end, the model can just do very simple kind of cuts and say everything that's left of this line is category A and everything that's to the right is category B. And yeah, so twisting and turning. And that's amazing. Not sure if that helped.

No, it helped me a ton.

Okay.

What is the most inspiring thing that you're excited about?

I actually, I don't like the word inspired. That's, uh, why don't we start there?

I think it's a great, great answer. Why don't you like the word inspired?

Ah, because it's kind of being misused. So I, I get that a lot, that people say, I was inspired by something, but in the end it's just more like, oh, I saw this and I copied it. And inspiration in my eyes is something that you see something and then you get a different idea. For me, it's the most important is to try to find inspiration outside the domain that you are in or that where you're trying to something to solve.

If inspiring is copying, then you're inspired about things that you've never seen before. Then how do you find things you've never seen?

Well, you have to give serendipity a chance to go to places where things are, well, where you never know what to expect. So personally, I love going to flea markets, old bookstores where like thrift stores were, well, You never know what you might get.

So for all of us to be more creative, it's really manufacturing randomness and serendipity with a curious look on it and making sure you're not looking at the same things every day.

Yes, try to burst your filter bubble. Try to— like, for example, I actively try now to follow people that are doing things that I have no idea about and/or are totally like in a different culture, even like people that I disagree with, but just to not kind of get stale into constantly getting confirmation. And because yes, there's often some, like even if it's a horrible person, they might have something that even out of context taken triggers some idea in me. But if I always just get confirmation in my beliefs, then usually that, well, I need challenge.

What area do you love the most?

Well, I love creating visuals. So I'm very, I would say I'm a very visual person. So everything from images, videos.

Specifically about images, what about it do you think is so interesting in the machine learning world as it relates to images?

Well, images are kind of a complex thing, right? Because an image can, well, there's the cliché saying an image can say more than 1,000 words. And that's true because there's a lot of information contained in an image. So an image can evoke emotions in you. It can— you can learn something from an image. An image can be interesting or not interesting, which is a question I try to actually— it's a core of my interest.

Well, how do you know what is interesting as a question to find interesting?

So there's something about images that cannot put a word on because it totally depends on what you have seen before. But it's definitely this thing like something you have not seen before or that in your understanding of the world does not make sense or does not fit together. This is, these are things that usually come to your attention because it might be dangerous. It might be something you want to eat. So the problem was for a long time, this kind of complex information, like how do you analyze an image and figure out if there's something in it that is interesting? Was not— you could not tackle using the classic techniques. But since neural networks are kind of more modeled after our visual perception system, they behave in a similar way. And suddenly you can actually turn an image into a bunch of very few numbers, and then you can start mathematically comparing them.

If I'm an artist or someone making something, is it possible for me to know what people are interested in now based on all the data? And and understand if what I just made is different enough to be interesting?

Well, you can measure if what you do now is different enough, is different to what is done around you. That is possible. If it's interesting, I would say you lack the information about that. So you would have to make some tests soon. So there are no universal models. That's the other thing. So usually you have to cater, train these models for the question you want to ask. That's the tricky thing.

It always comes down to asking the right question. Yeah.

And especially, for example, in neural networks, you also have kind of the separation sometimes between content and style. So there's this, like in a chemical process, you can kind of distill something like certain elements that make this up.

What is the difference between content and style?

Style is everything that is concerned with superficial layers like color and texture and this type of brushstrokes. And content is actually the semantic meaning saying like, oh, I see a dog there. I see a house. And neural networks are able to in some way tell them apart. And then it allows you to do things with that, like style transfer.

Is there style in writing?

Oh yeah.

Like commas or breaks or word structure or sentence structure.

Choice of vocabulary. Absolutely. So adverbs. You can statistically, like you take the work of an unknown author and map it, like analyze it by length of sentences, length of words, kind of size of vocabulary and can make a map. And then you distinguish prose from poetry and certain authors. So you get these clusters and So in some way, I believe, yes, you could do style transfer. I mean, you know, the technique of style transfer. So you take two images and you separate them into style and content, and then you exchange them, and then you can have something like a portrait look like a painting by Van Gogh. I believe you can do the same with text. So you could take the Bible and a detective story and then have the detective story read like it's written like the Bible and the opposite way. And, uh, well, I haven't seen that done yet, but I believe it will be possible. Or maybe somebody has done it and just missed it. Or with sound, sound, the same thing. So you can say something and, uh, but you will actually— that's happening already. So, and then you can sound like somebody else because again, like, your voice is kind of the words you say is the content, or kind of the low frequency, the structure, but then your high frequency and harmonics, that is what makes up your voice personality, and you can interchange that.

So when I saw Google Duplex, then basically what they innovated on with John Legend's voice, or it sounding like someone, is that they now have a machine that can copy style. Is that what happened?

Yeah, yeah. So there, this is just happening right now. So you can start singing like Aretha Franklin or so, maybe, or whoever you like. So So this is coming, definitely. Yeah. It will be interesting for pop music. I mean—

For sure. Thank you very much.

Hey, it was a pleasure. Absolutely. All right.

Thanks for listening. And thanks once again to the folks at LivePerson for their support in making the Machine Learning CAN episodes possible. Get in touch on Twitter @Assist. DMs are open. We're super interested to hear who you think should appear on the podcast. Machine Learning is made by Paul Ciuffo and Michael Elsesser for Limina House. Have a great day.