AI is creating never before heard sounds! 🎵

I have some kind of frustration with a lot of the way that machine learning stuff goes where it's like not fully attributed data sets. So we are trying to kind of do the hard tech in the opposite direction. All the data sets that we have are open source or we hired our friends or musicians that we know to create a data set that is fully licensed for this purpose of creating a generative model. And then I think when people are able to in mass create their own generative models, then there is a really big opportunity for creating value for those musicians getting paid for the generative use of or the licensing of that generator.

Big thanks to our partners Linode Fastly and launchdarkly. We love Lino. They keep it fast and simple. Check them out@linode.com changelog Our bandwidth is provided by Fastly. Learn more at fastly.com and get your feature flags powered by LaunchDarkly. Get a demo at launchdarkly.com this episode is brought to you by our friends at rudderstack and we're calling all data engineers to check out rudderstack cloud and start building smart customer data pipelines. Rudderstack is warehouse first. No more silos. Rudderstack builds your customer data lake on your data warehouse, not theirs, enabling all functionality of a CDP with more security and retaining full ownership of your data. It's open source and API first. Rudderstack can be easily integrated into your existing development processes and because they're open source you can see all their code so you don't have to worry about vendor lock in or black boxes. And best of all, they have transparent pricing. Stop paying your CDP a premium to store your data. Rudderstack is free up to 500 100,000 events and pricing scales transparently. From there, learn more and get started@rudderstack.com again. Rudderstack.com that's R U-D D E-R S T A C K.com.

Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning, learning and data science happen. Join the community and slack with us around various topics of the show@changelog.com community and follow us on Twitter. We're @practicalaifm.

Welcome to another episode of Practical AI. This is Daniel Whitenack. I'm a data scientist with SIL International. I'm joined as always by my co host Chris Benson, who is a tech strategist at Lockheed Martin. How you doing Chris?

I am doing very well. How's it going Today Daniel, it's going great.

I don't know if we talked about this. Are you a musician in your past at any point?

I have been a musician a lot in my past, but I'm not currently right now. But I love music, I dabble a bit.

So I'm pretty excited today that we have Yotam man who is a musician and programmer. He's the co founder of Never Before Heard Sounds which uses machine learning and AI for musicians, which is pretty exciting. Welcome Yotam, Good to be here.

Thank you for having me.

Maybe we should just start out to let people know how machine learning maybe has been used before for music or some things that people are trying out there. Just sort of set the landscape for us. How have people tried and maybe failed or succeeded in using machine learning in the realm of music?

Sure, I mean there's been I guess a lot of sort of statistical music generation for decades now. I wish I could remember this person's name. But automatic music generation sort of in the MIDI form using Markov chains has been pretty successfully used to imitate, you know, classical era composers going back from the 80s and 90s.

And midi, this midi, for those that don't know, that's like a representation of like is sort of markup representation of notes and lengths of notes and that sort of thing.

Right.

Sadly, I remember all this all too well.

Yeah, no, thanks for stopping because I might speak in some other music specific jargon. Yeah, MIDI is notes onset and offset. And more recently a lot of stuff has been in the audio classification realm. Certainly people want to know when piece of audio someone's talking or there's music playing or in music information retrieval, which is a massive field. So this is just all super broad overview. And then we'll get into this little niche corner that we occupy, music information retrieval, which is about figuring out what the chord structure of a song is, maybe even the genre of a song. This has also been going on for, for more than a decade at least. More recently sort of deep learning stuff has been in the realm of music generation. So really trying to get some really accurate sort of composition length audio that maybe isn't a specific style of an artist or a period or a location. And then the realm that we're focused on at the moment is in the audio generation realm. So forgetting the sort of the pitch MIDI format stuff and jumping straight into how can we make new and interesting sounds using machine learning?

I'm wondering, as I'm thinking about this, we sort of talked about MIDI a little bit and then we have like, you know, I think people are familiar with at least like, audio files. Like, they might have mp3 files or wav files or something. But in terms of AI models and these generative models, how is audio represented in most of these models in terms of data input and generally how it's processed?

Yeah, great question. I would say the MIDI to audio jump is a similar kind of leap as, say, like, text to image, just a substantially more data to deal with and much more complex models. So MIDI can be represented in a number of ways. You can think of it as if you've ever seen sort of like a piano roll or a music box where you have kind of all of the notes in one dimension and then time in another dimension, or you can represent it just as sort of discrete onset events and offset events. So there's a great piano transcription model called Onsets and Frames, and it. It is able to kind of reduce the piano transcription output to a pretty kind of compact format by just giving you the onset and the offset of the notes, as opposed to, you can imagine, sort of this 2D array of data is a much bigger thing to represent on the audio side. There is a few ways to do it. The models that I'm typically working with are outputting sample by sample. So audio, digital audio, like a WAV file, is an array of floating point values, usually one for each channel, and that's typically sampled at 44,000 times a second. So we're talking about quite a lot of data for things that are generating in the raw audio domain. It makes it kind of a challenge, especially a challenge to do it quickly because it's a lot of data to generate.

Yeah. I assume that it's not that satisfying to. To have a generative model where maybe you hit a note on a keyboard and then like seven seconds later you get the output, right.

Yeah, it's delayed gratification there.

I don't know that that would promote, like, a lot of jazz improv and that sort of thing.

Yeah. You can imagine how hard it would be to learn an instrument where, say, you sit down at a keyboard and you play for a minute and a half and then it's totally silent, and then you come back and you can hear what your minute and a half sounded like. It would be a pretty laborious way to learn anything. So specifically, the models that we focus in are, in the audio domain are real time. So they both need to generate faster than real time, and they need to have small enough little buffered chunks to be able to keep up with A low latency. So there's kind of two parts of real time audio processing. One is the real time, which is like it takes less than a second to produce one second of audio. That would technically be a real time model. But then a low latency real time model would be. As soon as you ask for the beginning of the audio generation, how long does it take to give you the beginning of the response?

You'd kind of have to be Beethoven, being the deaf composer that he was, to be successful. Otherwise, if you were having that delayed period all the way through there, that.

Would be a tough.

These models that you're working with, these generative models, I mean, maybe the first thing that comes to people's mind when they think of generative models is GANs. So generative adversarial networks, is that sort of the realm of things that you're working with? Or what's the sort of range of those types of either frameworks or architectures that you're exploring in the audio space?

We had some success with GAN based models. We actually made like a little website where we show off two of these models called GAN dot style. You drop in a YouTube link and then it replaces the audio of that YouTube link with the audio generated from one of these models.

That sounds exciting.

Yeah. So the two models that we got running in the GAN style are a choir model that's generated From I think 11 plus hours of kind of choral church music and a string quartet model which is from a university computer science music crossover department that they released a paper with this, with this data set. So those are both trained using GAN and a discriminator for the loss. Also had some luck losing the discriminator part. It trains a lot faster. And there's other kind of losses that we were able to use. I think if you look at the paper out of magenta called ddsp, you'll see really successful version of this sort of timbre transfer. The same kind of technique turning one sound into another. And they don't use a GAN approach, they just use different loss, FFT loss.

So I have an almost a side question because I'm fascinated by this and it's a use case for gans that we haven't talked about yet, which seems kind of weird at this point since we all love music and everything. But I can't help wonder what, what drove you into this? I mean, I guess you were doing music first and getting it, you know, how did you combine all of this to go do this? And what was the motivation to get you Going because as you've been explaining this, I've been wondering that along the way.

Yeah. How does someone end up here? Good question.

How do you end up there? Yeah. I mean, really?

And how can I end up in the same place?

It sounds like a lot of fun. He was much smarter about it than we were. Yeah.

Yeah. I actually never even really thought. Thought about, like, learning how to program or anything like that until I was kind of into my, I guess, junior year of college. I studied music. I played piano and I studied jazz piano. And that was kind of my whole focus. I was really interested in kind of music production techniques and different kind of things. But I had never expanded to the realm of sort of like entirely generative and especially not like, machine learning generated.

That wasn't just the thing that popped into your mind studying music in school.

Yeah.

So I was lucky that I went to a school that had this computer music department, which I didn't know was a field, and which is, you know, like a very vibrant field over the past many decades with a few kind of, like, bases all around the world. And one of the bases is at UC Berkeley, the Center for New Music and Audio Technology. And I just was sort of enamored by the way that people were making music there in this entirely new. Seemed like really innovative. Seemed like the stuff that they're making is like a huge departure from anything that you could do using different techniques. So that kind of led me down this route of, like, okay, how can technology kind of, like, make new music? How can it kind of keep progressing and pushing music forward? And I had done then many projects sort of in music technology, building synthesizers, building even, like, music generator systems, doing some generative music for video games, different things. And maybe about four years ago or something, I started to get more into, okay, machine learning. This is the next big technology innovation. What's this going to do for music? And so it kind of like, I just saw a huge, huge opportunity in all of the potential. And so about two years ago, I started a company with a partner, Chris Diener, called Never Before Heard Sounds, basically entirely dedicated to this question of, okay, what is AI ML gonna do for music? And specifically with the angle of, like, you know, there's a lot of people doing not a lot, but music generation systems, you know, like, for ads or for. I totally understand the, like, utility of that, but that, for me doesn't progress. Music like, that doesn't push the envelope of what's possible. It sort of puts music in this, like, utilitarian category of, like, okay. Music serves a function, you know, helps you study, you know, like works really well in this ad video and things like that, but it doesn't kind of like, you know, the way to progress music is to put those tools in musicians hands to build the instruments. Yeah.

So you're there in school, you're a junior, you have not previously done this, you discover this. But I gotta say, and I'm not just talking about music here, I'm thinking back to myself and probably most people. To leap into machine learning and to suddenly be able to be a practitioner in the space. That's a tall order. That is a significant leap right there. Can you tell us a little bit about, you know, you're this musician, you're doing this, you discover this, you get this exposure. How did you make that jump into being a practitioner? How did you accomplish that?

You know, I graduated over 11 years ago, 10 years ago. So it's been a slow transition. It definitely was not something overnight. First started with there was this great, you know, my like primary language that I got super into after college where it's like, it's all Python and Java and Lisp for some reason. So, you know, then the thing that I was most fascinated with is JavaScript because it's like you program in JavaScript, people can run your applications on like billions of devices and create language for that reason. So I got into the first thing that I looked at with machine learning was this really simple neural net implementation in JavaScript and I wish I could remember the name. I remember the author's first name was Heather.

I think I saw that. I don't remember either, but I think.

I rem from years ago.

Yeah, yeah, I think I remember what you're talking about.

If you know what we're talking about, let us know and we'll put it in the show Notes. That's right.

Yes please. Thank you. So it was this great compact library and just being able to see those things in this JavaScript is a fairly simple language. Being able to see those concepts like kind of laid out in that simple way, that was my start. And like in the beginning I wasn't able to really produce anything interesting or fascinating with this. But like I kept trying, I kept seeing that potential.

Foreign.

Deep learning evolving at lightning pace. It's more important now than ever to research the best options suited to your unique needs. This is particularly true when building custom systems and those systems that are GPU heavy. Not only do the applications running on the system matter, but your AI infrastructure and budget constraints need to be front of mind as well. PSSC Labs, which is an HPC and AI custom solutions provider based in California, has been creating high performance computing systems to meet their clients unique enterprise computing challenges for more than 25 years. And with cloud computing costs growing at astronomical rates, plus companies increasingly losing control of their data security, it is no wonder that enterprises and government agencies need to continually look for ways to take back control of their data. Solutions from PSSC Labs provide a cost effective, highly secure and performance guarantee that organizations need to reach their AI and machine learning goals. For more information and a free consultation, please visit pssclabs.com PracticalAI Once again, that's pssclabs.com Practically AI AI.

We were chatting before starting the recording about some of the projects that you're working on and I think you had mentioned this one AI vocal technology that you just released, which I think is really interesting and not only is it interesting technology, but I think it has a lot of elements in it that I think people may have follow up questions on in terms of data and the motivation and use and all of those things. So could you just introduce that? I believe it's called Holly plus. Tell us a little bit about that and how it came about.

Yeah, so I had mentioned this past project GAN Style and really the. For me, the interesting thing with these models that we've been developing is we can find cool data sets and we can train cool models, but really I think the big leap is in letting people train their own models. And now you have a generator of your specific sound. Your grandma's piano with the one squeaky note and the out of two not per register. Or your specific guitar playing technique where you slap certain notes and you pluck others. And obviously the most personal instrument that we all have is our voice. It seems like the voice models are really the most interesting personal models. So while we were developing Gan Cell, we're thinking about, okay, like it'd be cool to find some musical artist that we could collaborate with. And I always had Holly Herndon in mind just because she's done a lot of AI music projects. So I actually just basically tweeted at her musical Partner we were DMing and I was like, or no, actually he had first posted on our launch day, hey, is there anyone out there who could help us out with machine learning? You know, like any, any data scientists, people, ML programmers who could help us out with the project? I immediately responded in DM and I was like, yeah, what are you looking for? And he's like, well, we want to Build a website where anyone. Where we could train a custom model on Holly's voice and we could have anyone uploaded audio file, transform it into that custom model and then kind of like have the results. And we had just released this Gan style, which is a lot of the same mechanism. And I had already transformed one of her songs into the choir. So I sent it to him and he's like, I think his response was, holly and I are freaking out. Can we talk fast forward a few months? We had sort of repurposed this GPU backend that we had developed for or not repurposed, but augmented now to include this Holly Herndon model and trained a model from her, a bunch of recordings that she gave us of her voice. And she's got. One reason that this worked kind of successfully is that her vocal style is not acoustic. It's very digital. It's a lot of harmonized, a lot of vocoded stuff, a lot of digital glitches. And one thing that machine learning is really good at is producing some digital glitches. So, yeah, it worked out pretty well with her voice model. And then last week we released it to the world where anyone could drop in an audio file and have it transformed into Holly Herndon's voice.

I have so many follow up questions on this. I don't think we'll get to them all in our interview. So you'll have to keep me on track. Chris, maybe first off, you were talking about the voice being one of the most personal instruments that we have, but also a musician's voice. It's sort of like it's a big part of their livelihood. Right. And how they make money. So what are your sort of thoughts and what were your discussions with Holly and maybe others as you were entering into this project and the future of these types of projects around? Like, hey, I could, you know, I get Taylor Swift's voice and I can produce my own backing track and now I have my own Taylor Swift song and release it on and stream it on some service. And now I'm making money off of it. Right. So there's a lot of like messy stuff that could come up here. So what were some of your thoughts on that and what discussions have you had there?

Yeah, this was really for Holly Herndon and Matt Driehurst. This was the purpose of the project was like the age of this. Exactly what you're describing, Daniel. This is coming and they wanted to kind of control that story and offer a version of how this could go. So they have a whole mechanism for how Things that are created with this, you know, they're free and they're, they're able to be used, but the ownership is actually through like a dao, a decentralized autonomous organization. That's their version of how this kind of rights management could go. But I think that those kind of are the most relevant questions when you think about, about these types of generative models. Our version of this and how we've been producing and thinking about the future models that we produce is really like full transparency, full attribution, and also when we get to it, being able to kind of pay the artists back that went into this model. So I have some kind of frustration with a lot of the way that machine learning stuff goes where it's like not fully attributed data sets or like a lot of scraped data sets and things like that. And so we are trying to kind of do the hard tack in the opposite direction. All the data sets that we have are open source or we hired our friends or musicians that we know to create a data set that is fully licensed for this purpose of creating a generative model. So, and then I think when people are able to kind of in mass create their own generative models, then there is a really big opportunity for that being sort of, you know, creating value for those musicians, like getting paid for the generative use of or the licensing of that generator. So that's kind of how I see that potential future.

So Holly was a part of this process and maybe had a vision for the future of how this data was going to be used and potentially how her career could be benefited from it, maybe how her bottom line and her finances could be benefited from it. Whereas if I go to Spotify and just use some tool to capture a bunch of audio from an artist, they have no knowledge of that particular use. So part of it, I guess, is on the data side and maybe part of it is on how the model is, is released. I know we've been seeing a lot more models, even, you know, outside of this genre of models being released as APIs, where there's sort of more control in terms of who's using that for what and how that exchange of value is happening and that sort of thing. So yeah, do you see a lot of these models maybe in the future being released in that sort of API form? Because I could also just throw the model up on S3 or something and then anybody can use it to generate a voice maybe.

Yeah, true. For the time being, we are in full control of how these models are all used. They are behind an API that we control. They are on our servers and we are not yet at the place where we intend on open sourcing these models or their weights. Partially because of these reasons of being able to kind of control who uses it and give value to the musicians effort that went into this and kind of honor that by not just giving it away for free everywhere.

It's interesting is that this is an area like pretty much everything we've seen in the AIML space. That because it's a totally new way of doing things and clearly there will be lots of people. I mean, this is. I mean you're pioneering something that I think will be huge going forward. And yet there's the kind of AI ethical consideration that is kind of built into everything that we do. You know, we've had this conversation across so many topic areas and stuff, it's pretty cool. I mean, so you're recognizing that early on and there are questions that have to be figured out and like every other field and it sounds like you're taking a kind of a careful, you know, kind of respecting that, that process up front. How do you shape that as you as a pioneer yourself in this field. And you're looking at an industry where other people will start doing either the same or very similar things and exploring their own creativity. Just like we've seen in, in convolutionals with, you know, artwork creation, you know, in terms of visual things. How do you think that will roll out going forward to try to. To try to have a whole new industry in music that is taking advantage of this and merging it with existing approaches?

As a small company, all we can do is kind of lead by example and hopefully make this sort of public enough information that people might expect that from other companies full attribution of the data sets or that the musicians involved are paid. I guess part of it is really education of how this stuff works. I think a lot of times the story that's sort of spun is like the AI as some agent that's like, you know, like a magical character that's off in the cloud doing some magic up there. But you know, the story that we're trying to make clear is like, no, that's, that's actually not really how it works. There's a bunch of musicians, you know, there's kind of this condensed mathematical model and that is a generator that you can then play with. But in the end that model acts as a conduit between you, a musician and the data set musicians. So making those dataset musicians as kind of clear as possible, and making that narrative as clear as possible is our approach to it. So that when other data sets come out, I think the immediate question for consumers is hopefully, okay, well, where'd you get the data? Who's involved? What are their names? Were they paid for it? And those kind of things. And yes, putting a face on that data set is our approach that hopefully catches on.

And that data set that you gathered or worked on with Holly and her collaborators, what did that end up looking like? What data was needed to actually make this happen in the end, in terms of how much and someone could put in almost any kind of audio into this thing. Right. We actually just had a conversation, I think, in the last release episode about out of distribution input into models. And sort of here you've got this whole range of whatever audio could be that could come into this. So what did it take to put the data set together and get it behaving reasonably for a sort of wide variety of audio input?

It's the right question, which is like, how do you train something? And really it all just comes down to the data set. So for us, it was a few iterations. We had definitely a bunch of models that didn't really come together in the end. And the initial ask was for Holly and Matt was, okay, can you Give us roughly 2 hours of audio wav files that are self similar? And the self similar part is kind of an abstract notion, but we're like, okay, whatever is the Holly sound, give us those. So we had a few back and forths and we trained probably about a half a dozen models on different permutations of their data set. We built a few tools that do sort of data set analysis and throw out things that are too far or the thing that ends up being really far oftentimes in audio data sets is like silence. So automatic silence kind of trimming and things like that. And so, yeah, roughly about two hours of audio is what we've been using as our rule of thumb for these data sets. But it all depends on kind of how sort of self similar it is and then what you're asking for it to do on the output. So because this was something that we wanted to be able to handle pretty much any audio file that people throw into it. We also just trained it for a really long time to try and kind of get out all of those little weird squeaks and edge cases that happen when you ask it for something that it knows nothing about.

We deserve a better Internet. And the brave team has the recipe for bringing it to us start with Google Chrome. Keep the extension, the dev tools and the rendering engine that make Chrome great. Rip out the Google bits, we don't need them. Mix in ad and tracker blocking by default, quick access to the Tor network for true private browsing and an opt in reward system so you can get paid to view privacy respecting ads. Then turn around and use those rewards to support your favorite web creators like us. Download Brave today using the link in the show notes and give tipping a try on changelog.com.

Yo Tom, I want to, I want to return to a comment you made much earlier, which is that one of your big focuses in your work is real time audio processing. And you sort of describe what that meant in terms of, of like, oh, maybe you have around 44,000 samples per second and you're wanting to be real time and low latency. So what has that journey been and the tricks that you found and that sort of thing in order to actually reach that level of performance in a reasonable way that you can support. Because I'm sure there's a lot of people out there that are interested in that, you know, real time deep learning or AI side of things. Maybe not even in the audio space, but in, you know, video or text processing from users or something. What, what's your journey been in that real time space and what are some of the tricks and what you've had to learn in order to reach that performance?

Basically trying to, you know, we use convolutional models, those run the fastest in general and they also will give you oftentimes, you know, because of the different convolutional layers, they'll give you sort of a bunch of garbage, I don't know how else to put it, a bunch of random stuff on either end where there was padding added. And what we need is, you know, which is fine if you're generating. It doesn't make, you don't even really hear it or notice it if you generate say five minutes of audio. But if you generate a little tiny like say 256 sample chunk, where a large percentage of that chunk is going to just be this sort of convolutional padding garbage on either end. So there are a few different techniques to kind of deal with that. The most simple is you just sort of trim it off. The next, more complicated one is you do what it would have done internally, which is you pass the end back in as the beginning and sort of repeat that over and over again. And so you're always just kind of swapping things around. That's how we've been able. You know, most models aren't made to kind of stream in this way. What I'm describing is how you make these convolutional models stream. Most of them are just made for large batch processing and we need tiny batches that stream really fast. Another thing that we've had a lot of luck with in terms of getting multi X speed up, I think maybe like four or five times faster was when we converted everything to TensorRT. And so that really helped. That required also kind of changing the architecture of our model a bit to fit what I guess TensorRT had or has implemented.

Could you talk a little bit about the, about that, about making that conversion and what it was?

I mean there was some just basic stuff like the I used this library, now they actually have a replacement that I think Nvidia maybe is more officially supporting. But I was using this other library called torch to TRT that did the PyTorch to tensorrt conversion and it was a bunch of layers basically that it didn't have implemented. So I needed to kind of take a look at what their code was. For example, 1D convolution, which is used all over in audio, but it's a niche thing. I think when I submitted the pull request for like, hey, I implemented 1D convolution. This is what it is. I basically just copy pasted. It wasn't anything crazy that I invented. I copy pasted their 2D code and made it work for 1D. And the person who responded's question was what do you use this 1D convolution for? And I think it's just like most people who are doing image or video stuff aren't used to seeing 1D convolution. And that's probably why they didn't even bother implementing the converter. There was another thing that came up which was they didn't have specific kind of padding that we were using implemented. So I needed to swap out a bunch of layers. It wasn't pretty work. It was a lot of just kind of like grinding away. And there wasn't any massive epiphany that I can give your listeners. It was really just like, yeah, I went through it step by step and where things broke I tried to figure.

Out a way around. And so then do all these style transfer and real time audio processing models in terms of making that real time is a GPU required at inference time basically for all of these?

Yeah, for ours currently they all are. And that's what got us to actually like trying to get it on little tiny pieces of hardware Nvidia makes.

Yeah, I've seen pictures with like a little knob and such, which was really intriguing for me because I was like, that's really cool. I mean, I was wondering what's inside of that box, I guess was where my question was leading. Because if, if all of that's requiring the gpu, what ends up having to be in that sort of small box to make that happen? Because is it inspired by a sort of guitar pedal type profile?

Yeah, exactly. We're trying to meet musicians with the tools that they're accustomed to and the things that they have and what not a lot of people have is like a Linux machine with an Nvidia card and the specific version of Cuda inside and all this kind of stuff that. So we're like, okay, well what we could do is have basically a little computer with a little GPU and have everything basically pre installed on it. And Nvidia, they've been actually now, I think had it for a few years, but they've made some big strides with their Jetson platform, which is exactly that. It's like a Raspberry PI with a little attached gpu. It's not even a discrete gpu, it's like an integrated gpu, but it runs all of the Cuda libraries. It does give you a big speed up. We've been targeting that for our real time hardware offering and the idea is to get it fast enough and stable enough that we could have a consumer product that would work.

Yeah, and I guess if you had that platform and you know, that GPU integrated, it seems like in theory you could create a little sort of pedal type device where you could load a whole bunch of different models on it. As you release new models, you could sort of load them on or swap and that sort of thing. So because I think a lot of guitar pedals, it's like it does a specific thing. Right. Maybe this one would be sort of flexible in that way. Is that kind of what you envision?

Exactly, yeah. So I think that to me is the really exciting part with all of this, is the models. What can we train? What can we make a good generator of? I think we'll train a bunch of models. We have trained a bunch of models, different instruments and data sets that we've either licensed or recorded or open source. But letting people train their own model and then passing it over to their friend and saying, hey, this is saxophone technique that I was playing with. Why don't you try performing violin through it. I think that's like the real exciting future.

So when do you think, you know, beyond kind of the gimmick stage, when do you think we'll be going to like concerts, you know, major concerts at major venues and have that integrated in to the, you know, the performance itself and not as like, oh, we're using AI on stage. But you get past that and it's truly part of the art. It's built in as much as the instruments they're already using. Do you think that's sooner or do you think we're still quite a ways from that at this point?

I think if we have our way, then it will hopefully be pretty soon. Yeah.

Okay, good answer.

That's exactly what we're working towards. And that is, you know, I think that the AI is gimmick will hopefully fade away. And the AI is interesting, nuanced, you know, like wild sounds and integral.

It's integral to the art. Yeah, yeah, part of it as much as every other part.

Yeah, maybe. That brings up another question which I've been thinking about, which is, I mean like you say computers and synthesis and, and actually very complicated math and all that has been used in music for a very long time and synthesizers and other things. What do you think creative wise AI based techniques provide for musicians in terms of their own composition? And that sort of thing maybe different than computer based music in the past. What are the sort of qualities of AI generated music that maybe distinguish it a little bit from what musicians might have a feeling for already in terms of synthesis?

I mean, I think that there's a few answers. One is the big one is I don't know and I want to find out. Basically, Jimi Hendrix was not possible before the electrification of the guitar. Those two things kind of go hand in hand or you know, so much of what you hear on the radio. People, you know, young thug being virtuosic with. With the auto tune was not possible before. Auto tune was, was a tool. So I think really the most interesting part is like how people. What we've kind of laid out is like a utilitarian thing. Oh, I don't have a saxophonist, but I have a saxophone model. So let me use the saxophone model to generate a saxophone sound. But I think really the interesting part is where it breaks and how that is kind of pushed to stretch to its limits to make something actually truly new. So that's what I'm most excited about. I think another kind of thing that AI affords versus traditional techniques is more of a abstract. Like, call it sort of like a feature space. Like some say, like a saxophonist's tone is a really sort of like abstract nuance. Like a lot of things combine to create that specific musician and their specific tone. It's kind of way too many parameters to model, though people try. And there's a lot of really cool saxophone models, but AI kind of lets you jump that whole modeling part and just have these learned features. And then maybe you can even sort of distill some of the learned features, like maybe you can extract attitude from the saxophone model and maybe replace it with something else. And so this is another kind of really interesting, exciting area of how these models can be used that is a big departure from what we currently have.

Well, I know I'm super excited to watch all of these things unfold and also to get one of these little boxes to plug my guitar into whenever, whenever it becomes available. Well, thank you so much for sharing all this with us. It's been a real pleasure and I love how, you know, just to see the creativity that people are using AI for creative tasks, I think is really interesting. But there's also creativity in how you've gone about applying AI, which is is amazing. So great work and thank you so much for joining us.

Yeah, thank you for this.

Thank you for listening to Practical AI. We have a bundle of awesome podcasts for you@changelog.com including our brand new show Ship it with Gerhard Lazou, a podcast about getting your best ideas into the world and seeing what happens. It's about the code, the ops, the infra, and the people that make it happen. Yes, we focus on the people because everything else is an implementation detail. Subscribe now@changelog.com shipit or simply search for ShipIt in your favorite podcast app. You'll find it. Of course, the Galaxy Brain move is to subscribe to our Master feed. It's all Changelog podcasts including Practical AI and Shipit in one place. Search changelog master feed or head to changelog.commaster and subscribe. Today, Practical AI is hosted by Daniel Whitenack and Chris Benson with Music Brakemaster Cylinder. We're brought to you by Fastly Launch Darkly and Linode. That's all for now. We'll talk to you again next week.