#34: AI Music is Cooking

Well, this will be fun. We're talking about music. Who are you?

I'm Edward Balasanian. Good to see you again.

So, you know, music has gotten really competitive since the last time we talked. And thanks for doing that first. You were my first interview here on Unaligned and I remember. Appreciate that. Elon Musk said it was a. A kind of video he wanted on X, you know, so I guess we got to do more.

Exactly. I think we went. We went to like 60 million views on that one, didn't we?

Back in the heyday of X. Yeah. Now everybody's going somewhere else. You know, Blue sky or Threads are Mastodon or whatever. It's like, oh, man, I gotta chase people all over the place.

Now you say back in the hated. That was just like a year ago. It wasn't even that long ago.

You're still moving pretty fast. I'm watching it X all day long. So. Yeah. So what's going on in your world? I mean, that's. Your space has gotten hyper competitive. I mean, you got Suno and Udio and I got a whole list of companies in the AI music space. What's going on in your world?

Well, look, I'm almost afraid to wake up in the morning because I feel like another AI music company will have launched by time I went to bed and woke up. So the pace is. The pace is incredible to see. I love seeing all the innovation. There's a lot of different takes on how to approach AI and music. We took our path and we focused more on training models to do arrangement, meaning to emulate what a producer does. Whereas other models like Suno and Udo have taken an approach of essentially copying finished music, or if you want to be cynical, plagiarizing finished music. I think we're going to see a lot of shakedown happening in this industry in the next year or two. So. Especially with the lawsuits that are already filed or soon to be filed against companies that trained on copyrighted material.

Yeah, we have Ayelet here too. Ayelet's a famous PR person. Got me many companies, so I appreciate what she does as well.

Thank you.

How should we focus this conversation? Because, you know, there's a lot of people who say, oh, AI music sucks, you know, and they're usually old people like me who went to, you know, Metallica concerts or Elton John concerts or something like that. Human, human made music. And this whole new AI music world is coming along where you can prompt to an engine like yours and get some. Get some music out of this. I find that, that, that kind of talk is crazy because I don't think, I still don't think you're going to take on Metallica or, or, you know, you know, any of the human made music. Where do you think this is is right now? Because I, I'm really excited about AI music because it's fun to use. You know, you, you can put a funny quote in there or something and it'll build you a song around it and it's a lot of fun. Or you can just say, I just need some beats, you know, so I can work and it generates you some beats. How do you see this argument going down right now?

Well, look, you said something really important, which is, can we take on Metallica? And I think the better question is why would you want to? Fans have a parasocial connection with Metallica. There's an artist that goes on tour, there's an artist that is vocally promoting it on social media, and that is as much a part of what we consider to be songs as the actual music itself. So I don't see AI going on tour or getting on social media and promoting a song that it wrote. And I do think that parasocial connection is unique to music. You don't have that connection necessarily to visuals that you see online, like a picture that was taken. You don't necessarily know who the photographer was. But with music, the parasocial connection is critical. The other thing that I think is important to distinguish here is the difference between songs and music. We see ourselves as in the business of music, and music for us is something that accompanies existing products that have value in and of themselves. We're not trying to create music that'll stand on its own. That is the thing that you're going to consume. So we make music for videos, we make music for commercial spaces. We make music in places where human producers might not be able to provide the kind of automation that we can. I'll give you a quick example. We're working with ad tech companies right now who are going to be scoring ads using Amy's Generative music platform. But each time you view the ad, the music's going to be different and it's going to be created on the fly based on your demographics, where you live, what language you speak, what type of music you like, et cetera. Humans can't do that. And I think that's where the application of AI is really exciting, where we're filling this void rather than trying to plagiarize the creative works of artists.

Yeah, man, There's a lot to talk about. You mentioned it. There's one angle at this that's, you know, creating music for commercial context. Video games or advertising or, you know, a doctor's office that doesn't want to pay for the ASCAP licensing fees. Right. They can prompt the engine and have a whole bunch of music created for their doctor's office or whatever. Do you see that as a big part of your business? And how do you think that's shaking out?

Well, we have four products right now that all are based on the same engine. One is our live streams, which are basically continuous music in over a dozen different genres. They're very easy to integrate. You just click a link and you've got music. Then we've got our player app, which is designed for more interactive music so you can actually interact with it in real time and personalize the music. Then we've got a new product that we just launched called Amy Sync and this is a tool that lets you add music to video. We analyze the video, we break it down into scenes, and then we identify the right type of music per scene and we let you interact with the plan. Those are all those products we see as enterprise play. Like the live streams are for commercial spaces, the interactive music player is for commercial spaces that want more control over their music. The Amy Sync product we're really excited about because the market size for video, if you consider all the different mediums of video advertising, social media, film, it's enormous. I mean, there's 34 billion videos uploaded to TikTok monthly, I think, or sorry, 34 million 15.8 billion minutes of video uploaded to YouTube alone. Crazy annually. And there's almost all of those need music. And there's a huge problem there because catalog music is really all that's been available. Or you can go into a song gen tool, but they make songs. If you've got a 30 minute video, a song isn't going to work now. And I think that's where we really have found our sweet spot.

Particularly if you're trying to design an audio signature, you know, or a feeling of some music underneath your videos on TikTok or even on X or Instagram.

Right. Who.

This is a weird world. Are you aiming at one of these four features at professional musicians at all? Because I see that they're starting to use a. Even the Beatles was using AI to separate their instruments and their singing from a crappy audio tape on their latest song that John Lennon recorded decades ago and they were trying to make that sweeter with AI, all sorts of new musicians, you know, from Skrillex or whatever, are using technology to create music. And I assume, and I know they're using AI to create all sorts of like beats underneath their songs or, or even vocals. Right. What do you, what are you seeing happen from. From the professional side of the house?

Well, that's not new. I mean, professionals have been using technology since the DX7 came out. So this is, this is just the acceleration of a trend that you've seen. Music is part math and it's part art. And that's one of the unique things about this particular medium for us. You know, like our Amy Sync pool is really targeted at professionals, but I would say more professional video producers who don't want to be mired in the music side of things. And we provide them with a way to very quickly add music, they get full control over it. It's not a prompt and praise system. It's really more of a generative, automated platform for creating music. So we put together the elements of music for you and you can see it and you can adjust it as you want. That gives you a lot of control. And that kind of control, I think has been sort of missed on the industry because you were either a video producer or a music producer, and very rarely both. And that skill set is pretty unique. So we're making it easy for video producers to express themselves musically over any length of video. You can do 10 seconds or you could do a two hour movie with, With Amy Sink. Yeah.

You know, years ago I went to Coachella a few times and I noticed there a generational shift in music. I noticed old people would go, so let's set this up properly at Coachella. It's a music festival with like seven different stages of different kinds of music. You know, there's rock and roll, there's a, you know, more jazzy and more. And then there was electronic music, which was in a place called the Sahara Tent. And the Sahara tent was having the biggest party. You know, that was a crazy place to be. You know, 20,000 people dancing, throwing things in the air, holding flags and all sorts of craziness. And my friend Thomas Hawk and I had these long 600 millimeter lenses and we were shooting people, you know, out on the field and having fun. And we kept noticing old people would drop their kids off at the Sahara Tan and then turn around down and go to another venue, like go watch Arcade Fire or something like that. Right. And so I know I started interviewing these people, I started grabbing them, like, why Aren't you going into the tent where the party is? This is obviously more fun than Arcade Fire. You know, everybody's sitting around like this in the field at Arcade Fire, but in the Sahara town, everybody's like, whoa, crazy people, right? And that turned into a panel at the music and tech conference that Brian Zisk wrote Ran, where we talked about generational shifts of music. Because I, I noticed these old people didn't like the new music. And they gave me six reasons why they didn't like it.

They.

They told me, oh, they're not playing instruments. They're. They're doing all their music on a computer. That's not music, right? And. And then they would say, oh, there's. There's no lyric, there's no story. You know, I, I want to hear a story being told when I'm listening to music. And they said, oh, I didn't like the lights. The lights are too intense, right? Because when I went to the who for the first time, you know, we went. They had incandescent lights. You know, they were pretty bright, but they weren't like these LED lights. I mean, at the Sarah 10th, there was 110 LED screens around you, some of which were on robots and all sorts of crap like that. And there was just high intensity flashy lights. In fact, there was warnings on the side of the tent. Don't come in here if you have a certain kind of epilepsies, right, because you're gonna. You're going to get shredded and so on and so forth. Anyways, there's this generational shift of music that the younger people don't care about, you know, playing a guitar. I actually talked to Slash about this and he says, oh, yeah, that, that's why they have to do that. There's nothing to watch when you're playing your music on your computer. So they have to put a lot of art around you and a lot of lights, right? Lasers.

And there's some truth to that.

Yeah, right. And what you're representing now is a new generational shift in music, because now it's not even the computer. You're not creating the music on the computer like Skrillex did for a long time. You're actually prompting and, you know, creating music automatically. This is a whole new world. By the way, at Coachella this last year, there was a performer that was a completely digital performer. There was no human on the stage, and everybody was still dancing in the Sarah Tan. Sarah, so what do you think about this generational shift of music? Are you seeing it? Yourself that new people are coming along to use music in a new way.

Oh, for sure. Look, let's just call this Sahara music. Electronic music.

That's what it was. It was new, bleeding edge music. Yeah.

I think the difference is. And electronic music, by the way, I think it's the number three genre in the world. It's the fastest growing genre in the world. It's much more prevalent outside the U.S. than in the U.S. but it's very quickly becoming a mainstay here in the US as well. I think the key difference is going back to songs versus music. When electronic music is designed to be a journey, you can listen to it for four hours. The structure of the music over time is designed to engage you but not exhaust you. It's tough to listen to pop music for four hours. It's tough to listen to singer songwriter music for four hours. It's partly because of the lyrics. Lyrics really engage and pull you out. That's not to say that electronic music doesn't have lyrics, but when you listen to a pop song that's three minutes long, it's kind of got a beginning, a middle and an end. And you engage and you pull back and then you get another one. You engage and you pull back. And that kind of back and forth with the music tends to be exhausting. Whereas electronic music just kind of ebbs and flows over time. And I think you'll see a lot more that I was reading this study the other day that. And I wish more restaurants knew this, by the way, that was saying that if you play vocals, lyrics in your restaurant space that are in the same language as the patrons of your restaurant, they're going to yell over the music. And it makes sense because your ears are competing with the vocals from the music and you end up having to yell over the music. So the right thing to do is to back off on the vocals. Play electronic music that engages but doesn't pull people out of what they're there for, which is to be with their friends and eat food.

Yeah, we have this argument in my house all the time, you know, because I love playing music and I like it loud. And everybody's there, there not to listen to music. They're there to talk to each other. You know, turn the music down.

Scoville. I think I get that one all the time too. Yeah.

Let's talk a little bit about surround sound or spatial audio with Apple music called spatial audio. I've been really interested in spatial audio or surround sound for a long time. I went to the Virginia Tech augmented reality research building. They have a cube there that's a big building, like 40 yards across. 40 yards across. 40 yards with 1600 speakers inside this building. It's insane. The sphere, it goes even way further. They have, I don't know, tens of thousands of speakers in the sphere in Las Vegas. Have you been there, by the way? It's freaking insane.

I'm going taking my family for Christmas, so we're going there. We're going to Las Vegas over Christmas. I know, judge me if you want, but that's where we're going to be.

No, I've been to Vegas 150 times, so I can tell you where to go in Vegas. Quite an interesting audio experience because it has so many speakers and they're using a special kind of music that's putting the sound or the sound because there's elephants and other things on their movies and stuff. Anyways, I love this kind of new genre of music I collect. I've collected a hundred thousand tracks of spatial audio on Apple Music. I really care about spatial audio. I think it's the future of experiences. But most people aren't there. Most people don't have a sound system that can play this or an interest in it yet. Where do you think this is going? Because I assume your service is still creating stereo music, two channel music. Right. Are you thinking about a service that would, you know, bring in more tracks of music so that I can be surrounded with sound? Tell me what you're thinking is there? Well, so our.

You can think of, Amy as a middleware for audio. And essentially what we do programmatically is what a producer does when they sit down with Ableton or Pro Tools. So they take disparate pieces of audio. Sometimes they pull something down from Splice or Loop Masters, other times they'll make their own sound. Maybe they'll take a sample and use an analog synth and create their own sound that way. Maybe they'll pick up an instrument and stomach. But in the end, they have all these bits and pieces of music and that they assemble and arrange to create the music that you listen to. When you make spatial audio, you do the same thing. Now, because we're able to program music from the bottom up, we're able to move music around at the instrument level. So we have a.

So you can say put the horn over here, put the singer here, put the drum over here kind of thing.

Exactly. And we've done this in an immersive theater space in Austin that I built that has speakers all around it. Nothing like the sphere. But because Amy is in control of the music from the bottom up, we can put any piece of music any place that we want. We can do combinations of instruments, we can move individual instruments. That's part and parcel to how the platform works. Again, we didn't train on finished music. We're building it from the bottom up so we get much more control over how we express it and how we also articulate it into a medium, whether that's a podcast or a video or the immersive theater space in Austin.

This really is important as we move into, you know, spatial computing with like an Apple Vision Pro or meta quest 3, where you can actually put sound all around you, you know, in, in 3D space. So as you're moving your head, it's locked to like my printer could be as an instrument. Right. You know. Yeah. And just stays there even though you're turning your head. Right.

And.

And now we need a new kind of music to really provide a bleeding edge experience for like an Apple Vision Pro.

Right? Yeah. You just said a really important word, which is new. And this is one of the challenges I have with this idea of using models to essentially copy existing stuff. You're looking backwards. Right. You're essentially training on the past. And one of the big challenges that these monolithic models are running into, by the way, we have eight different models that work in concert together to do the mixing, the masking, the ranging and the real time production of music. One of the challenges with these monolithic models is they're going to run out of data to train on. And I've seen numerous research papers on this. If you train a model on its own output, it'll eventually generate noise. You can't. You need new material and that gigs over. You're not going to really go and plagiarize all of Spotify anymore. So I think models and technologies that are focused on creating new, creating new as opposed to imitating the old are going to be the ones that have the staying power.

Yeah, this is going to be really a big deal probably in 2026 when there's a whole bunch of new AI driven glasses coming out, devices that you wear on your head and your ears, right. That do this new kind of immersive all around you video. I mean, Apple's playing with it. Apple just put up a video with the weekend in Apple Vision Pro and it's a, it's a wraparound experience. It's 3D. You're like right on the nose of the weekend. I mean it's Insane quality.

It is.

I don't know if you've experienced this yet, but if you go to the Apple Store, go and try that. It's insane. Right. And people who, who have vision pros are all like, wow, that's crazy. Right. So this is a new genre of a new world that we're heading into, where we're going to be inside the music video. Right. And it has to sound like that.

Yeah. And you're seeing, I mean, one of the interesting things with the, you know, with what Mark Zuckerberg has done with the glasses that that Met has been putting out is the price point has come down significantly, which is going to make this much more accessible. And then you'll see a rush to creative, to creating new mediums for these types of displays. And I'm, I'm all for it. I mean, I think it's, I think it's super exciting to see us express ourselves in new mediums using old school stuff like music, you know, where all of a sudden that music gets a whole new life in these new mediums.

Yeah. So what's the bleeding edge right now? What are, do you have a new version come in? What's happening at the bleeding edge of what you're trying to do with your company?

Well, I'll tell you one of the things that we did that's really interesting. First of all, we built this programming language called Amy Script, which is essentially a way to codify music in code that didn't use AI, it was just strictly scripting code separately. We had models that would ingest recorded audio. These are bits and pieces of audio and they would understand every nuance of those little bits of audio. So the code could essentially arrange, mix, master and produce music. We've now essentially taken the output of those expert systems and fed it back into our own model. So our own model can now replicate what the expert system was doing. And we, we completely circumvented the whole copyright issue with this because we're training on our own output, but our output's an expert system. So we've created a really exciting way to essentially continue to improve this system without needing any external finished content to train on. You'll see that in Amy Sync, which is our tool for adding music to video. We're able to take arbitrary length video and score it with music. It's really good. And it's not just the music, it's also voiceover sound effects. It's essentially the whole audio middleware. That product I'm very bullish on. We just Put it out in the wild. It's a early alpha product. We're now, we just actually not supposed to speak about this, but I'll give you kind of vaguely what it is. We're working with an ad tech platform that's going to be using this, as I mentioned, to dynamically score ads with the full audio layer, voiceover music and sound effects as well. And it's really cool because we can look at a video and see a door shutting on a car and add the door shutting sound. We can add in music that has sentiment analysis in it based on the video itself. You can't do that if all you've trained on is finished song. You just don't have that kind of flexibility in the way you construct the music.

Yeah, this is a big trend coming. Hollywood has funded a company from USC Cinematic Art School called Corto Quarto AI and they have built a database of how humans work. In other words, you know, what are our preferences and all sorts of stuff. I mean, it's crazy. And everything in movies, like everything. Everything about human beings and everything about movies. All the Hollywood studios have funded this company. So it's a, it's a big deal. It's coming next year and it'll enable people to create personalized media, personalized movies or like you said, ad. Right now they're doing a bunch of ads too. That sounds like it might be a competitor for you at some level.

Or a partner.

Or a partner. How do you see this personalized? Because this is going to be a big trend, right. Where you visit a webpage and the web page is built for you based on who you are and what you respond to. Right. Where do you.

I think there's going to be a fundamental change in the way we look at apps in general. I mean, I, I don't think apps will exist. I think an app will be a dynamic interface created by a model based on all the different parameters at that moment. You don't have to try to prescribe what the app should look like. Let's just go back in history. When I was at Microsoft in the early 90s, we were building software that wouldn't get in your hand for a year. We'd release to manufacture. We'd go to a CD and get pressed, go to a box, then from the box, sit on the shelf and we wouldn't get feedback on that for a year. Now everything is compressed. The second you open the app, it's going to dynamically create an interface. Your responses are going to change the way that interface adapts over time. So we've completely compressed that timeline that, you know, all software developers love. The feedback loop is going to be.

Instantaneous, and you're providing an API for somebody else to study the human being, right? Because I've interviewed Luma AI, which is watching your voice for emotion and eventually your face, right? If you're. It knows you're angry, Right. If you're smiling, it knows you're happy. Right? And therefore it can feed back that into an API like yours and say, hey, this is really getting this guy to dance and happy. Right? Give him more of that. How do you think that's going down? And what's the next year looking like for you?

It's all about data. To be honest, the creation of these models is no longer that interesting. It's really more about where and how you can acquire unique data. If you've got unique data, you can escape this challenge, which is that the data has already been consumed. The entire corpus of human knowledge has basically been sucked in to these models. So if you have access to unique and proprietary data, you're in a really good position for us. You know, like, the ability to be able to see how people engage with music. And that feedback loop at a. At a very microscopic level is huge. And if you tie it to other mediums, like movies or ads or commercial spaces or even podcasts, now you're able to adjust the music dynamically based on sentiment that's being collected in real time.

Wow. It's a whole new world coming at us. And it's. I mean it. In 18 months, I'm going to have a new pair of Apple Vision Pro on my head, and we're going to be in a new kind of Holodeck, a new 3D environment all around us that we talk to. And put a purple tree over there, and boom, it'll put a purple tree over there, right? And it needs music, right? The tree needs to dance and have some music on it. So the purple tree could call your API and say, we need some jazz music now. Right? Something like that, right?

Yeah.

Mind blowing. Totally excited about this feature. This is like crazy future coming at us.

I've been waiting for this for 40 years. So, yes, I'm very excited about it. Look, the big difference between now and 20 years ago, which I think is when you did an interview with me first, you were in your home in Southern California. We had these ideas. The algorithms were arguably not that far away. It was really all about the compute power and the Internet, the two things that Made all this information instantly accessible and the compute power to process it. That's what changed. And fast forward to now. You can go and download the entire corpus of human knowledge in days or hours. Even you couldn't do that in the early 90s. Now you can. And we've got compute power that can process it. So those two things have dramatically accelerated everything.

If I'm developing a holodeck and I know Dylan is a researcher at Unity Labs billing one of these, and he's not alone, and I would be looking at putting a music service into such a thing, I would be looking at latency and cost. How do you compete on those two things?

Well, look, this is an area that I think we are actually at the top of the pile. We have the lowest cost in generating music because our models generate code and our code generates the music. So we're, we're executing highly optimized typescript code or instead of inference. So we're able to generate, I'll give you an example. We have these continuous live streams of music. It costs us about 5 cents per month for 30 days of continuous music of inference. Model will not be able to compete with that. So we're on the cost side of things.

So there's no Nvidia cards involved in your life right now.

Well, there are, but not on the production side.

Got it.

We're very smart about where we apply the AI and it's on the ingest and understanding of music and the arrangement. But that arrangement is expressed in code, so we're able to execute that very quickly. And the other benefit that our model has is because we're building from the bottom up, we are able to adjust in real time at every element of the music. We don't have this challenge of, okay, we spit out a song, now we got to go spit out another song. We can actually dynamically create the music on the fly, just like a DJ or a producer would if they were playing live.

So if I was a video game producer and I was taking people through, you know, some sort of multi stage dance party, like a Coachella, as you walk from stage to stage or from room to room, the music has to change. Right. And so that's what you're talking about, is it? It could flow from place to place and be seamless and really nice experience, right?

Yeah. If a producer, if you give a producer a movie, what are they going to do? They're going to analyze every frame, they're going to look at it and figure out what's happening. And they're going to create music to do that. We can just do that in real time. You can do that with the same level of automation, if you will. I should say production as a producer would. We're able to do it using models and code. So the. It's the same concept, right? You're analyzing the video game or the, or the movie and then you're generating the music in real time to match it. I'll just say one more thing on this too. There's a whole industry that was meant to support producers. These are companies that make samples or loops or one shots or vocals or even vocal generators. And they're really threatened by this idea of going straight from prompt to finished music. We're kind of the savior of these businesses because they can all plug into our middleware. We can pull in vocals or samples or loops. We're really good at pulling in disparate bits of audio and turning it into beautiful music and doing that in real time as well.

Got it. Do you have a consumer play? Because the consumer is just sitting on their phone, you know, want a cute song to send to grandma, you know, with some funny lyric or something like that. Are you going after that or are you going to leave that to the Pseudos and the Udo's and that kind of thing? You know, there's a competition in this space.

It's like, wow, there is, there's. There's two pieces to this. We're not interested in making songs. It's not something that we, well, we, we emotionally don't have a conviction about that. We feel like songs are best left to Metallica and we don't want to compete there. We also think that the best way to compete there would be to plagiarize and we're not interested in doing so. That's part of it. The other part of it is consumers just don't pay for music. Historically, they haven't paid for music. There's a billion songs on Spotify. I think 20% of them have fewer than 100 plays. This isn't a problem of needing more songs. Yes, I do think it's fun to express yourself for your bachelor party's meme and you want a little 10 second cute song for it. But I just don't think people are going to pay for that. And I don't think outside of that there's really a business. So just to answer your question, more specifically, we do have a consumer app. It's free. You can download it, you can play with it, you can put continuous music on in 14 different genres. The music's good to great. It's really fun to interact with. We don't have any ambitions of making money on that and we'll continue to support it. But for us, that's really about getting feedback. When you hit thumbs up and thumbs down. Our models get smarter about how to mix, master and arrange music. So it's really a data play for us, more so than a revenue opportunity.

Is there anything else we need to talk about today? I'm not a question. Then we could tell stories about going to concerts or something like that.

I'm happy to give you a demo if you'd like to see a demo of this. Absolutely.

Yeah, let's do it. I'd certainly like to see this professional side of the house, you know, because consumers can go to Udo and Typhoon and get a song or go to your consumer app and get a song. I'd love to see how they how your code based system works.

It's like, yeah, let's do it. All right, let's see if this works. I've never done this on Chrome. Okay. On the left is this new tool that we just released called Amy Sync. Are you able to see it?

Yep.

Okay, so this bigger? Let's see.

Nope, I guess not. I don't know how to make it bigger anyways. We'll just deal with that.

Okay. Here. Well, maybe I can make it bigger on my side. Okay.

Maybe you can pin it because I know Edward is pinned at the moment.

Let me unpin him. Yeah, There we go.

That's it.

All right, thank you.

So this interface that we built was just to show off the API. So behind this is an API that's able to take in media like videos. It'll analyze it and then it'll suggest a plan for how to add music, voiceover and sound effects to this video. So let's go ahead and pick a video here. Oh, tab here. So I've got a few videos here. I'm going to take one that I like, which is this Maserati Ad. What's happening right now? We're uploading the video to our API. And when I say API, this is how ad tech platforms or even applications that need to interact with video content and add music to it, would use our service. We will have a version of this for freelancers, professionals and independents who want to score music, score videos with music. But this mostly is for demonstration. It's actually live. You can go to Think Sync, Amy FM and play around with this. We've opened it up to the public. What's happening right now is this video is being analyzed. We have proprietary tech that looks at every frame in this video and tries to get an understanding of what's happening in the video. We then break it down into scenes. And these scenes are meant to be thematic sections within this video. Once we've got an understanding of the synopsis of the video and the individual scenes, we'll then recommend a plan. And you'll see the plan on the right hand side. This is where you really appreciate how Amy thinks about music. We build the music from the bottom up and because of that, we're able to show you that bottom up structure and let you interact directly with it. You can see here this plot summary. We essentially described what this video is and we can also extract sentiment from the video. We can understand how the video evolves over time and how the music should correspondingly evolve over time as well. You can see on the right here, we've broken it down in the scene. So there's seven scenes total for this video and each one has a voiceover added to it. Now that's optional. I can remove it if I want. So in this case, this voiceover is added from the perspective of the brand. So in this case, Maserati. So let's go ahead and tinker with this. I'll choose techno just because this is the guy in tattoos, so maybe he likes techno music. He's going to go to the Sahara tent right after he's done. And you can see here I can add and remove elements too. So I can pop in here, add different pieces. I get full control over this. Now we're adding in the ability to do introspection here. So I can say give me a rhythm that's bright or give me a piano chord progression over four bars that's in C sharp. So I get full control if I want it as a music creator. And if I don't, I can just let Amy do its thing. So let's hit produce and see what happens. I also want you to note too, like there's a scene break here at 13 and a half seconds. And that's based on the analysis of the video. You'll see here at 13 and a half seconds, it's right when he gets into his car. And then we go into this driving section. So we're really smart about understanding where these scene breaks happen and also adjusting the music to meet with that. I'm going to hit produce. It's basically real time. So it's about a minute and a Half to produce a minute and a half video, which is how long. But while that's going. Let me show you. Let's see, I've got a couple.

This is already mind blowing. I mean, I remember talking to the CTO of Pandora and he had a room of humans who did some of this by hand. You know, listening to music and trying to figure out what the timber of the music was, what the content of the music was, you know, what kind of music it was and all that. This is all automatic. Down in a minute and a half. It's just like, wow.

Yeah, exactly. And there's a lot of work that we save people. So this isn't about replacing people. It's about accelerating people and giving them more creative expression without needing to be experts in the medium of music. So while that one's running, I've got this other one that I'll show you here. And this is, this is a Dropbox ad. So I did something fun here. I added in voiceover and you'll note that. Let's just play it. You'll see. Welcome to a world of seamless file sharing. Installing Discover Dropbox. Great. Happens at 13 and a half seconds, coincidentally at the same time. But look how precise this is. So we're going to get this geometric shape the spinning disk is about to kick in. With Dropbox, your files are always within reach wherever you go. And at the end here, I'll just fast forward here. Experience effortless sharing and collaboration. By the way, this was all automated. I did nothing. I just uploaded the video and Amy did its thing. So if we go to 57 seconds.

How did it make up the. The audio overlay, the. The, the words that we're hearing?

By analyzing the video. It analyzed.

You didn't send it a text file? You didn't tell it what Dropbox is all about or anything like that?

Nope. Yeah, it's. It's really powerful. So you can see here. Look at the time. Welcome to work in progress. So this, you know, this is why this is going to be really valuable for automated ad engines that need to essentially iterate really quickly through different copy, different music until they find exactly the right medium. One thing to remember too, we were talking to an ad company. They were doing one ad a month for their customer that were highly produced. Now they're doing a hundred and they're targeting it to micro audiences to see what sticks. You can't do that with a budget of $500,000 per ad. You need to be able to incrementally iterate through these things quickly. Wow. All right, let's go back and look at the other.

How much does this. Just to do this one ad? If I, if I had a video like this and I wanted to, you know, put down some audio, how much would I have to pay you to do this?

Hang on one second. The other ad, just, the other one just finished. It started kicking off on me. So we, we haven't released pricing on this right now, but it's going to be disruptive where, you know, our, our ability to generate music is incredibly efficient and we're going to pass that along. So you're going to see our pricing. Not only is it disruptive in the.

So a few dollars or a hundred bucks or. Because if you're saying disruptive, it's not a thousand bucks, you know, you're, you're getting down the price where it's real affordable for people to do this at scale.

Yeah, absolutely. We're gonna, we're gonna have three different models. One is going to be pay per minute, the other one's going to be pay per month. And then there'll be an API driven model that'll be probably driven through enterprise deals where there's a quota per month and then there's overage charges. If you go over that, you go.

Over that amount and can you force it to give different. Like if, if. My viewers in India, they probably like a little bit different kind of music, you know, Bollywood style music, you know, versus kid in California, like, you know, something like Skrillex or something, right?

Yeah. So let's. Is the Dropbox video on your screen right now? I'll show you.

I don't see it. I see the Dropbox video, the first one you showed. Yep.

Okay, so I can very easily just go change the voice over to Spanish and it'll change the voiceover to Spanish. This is exactly what we're doing with the ad tech platforms. If you're in Miami, you might get reggaeton and Spanish. If you're in Boston, you might get hip hop in English. So the ability to really take this kind of first customer data and hyper personalize the content that we're creating using music is really powerful. I'll say one other thing to this too. We did a bunch of studies in a previous life of mine where we showed video to customers and we modulated the video and the audio. So we showed bad video with high quality audio. And we showed high quality video with bad audio. People could not watch when there was bad audio, but they could watch when there was bad Video, it just shows.

You how powerful TV people have known this for a long time. You put better audio, you can have crappy video, nobody cares, but you have crappy audio, everybody's like, I can't listen, I can't watch.

Yeah, it's fascinating. Let me show you one more thing on this. I prepared this one in advance. We're now in the process of licensing premium vocals from labels and publishers. What this means is you're going to get memorable vocals, the actual original recordings of the REO Speedwagons, the Metallicas, or even more contemporary artists. Because it's the original vocal and because of the way Amy works, we're able to ingest that vocal, understand it, and then put it in to these videos in a way that makes sense. So check out this video I produced for lacoste. So that's Amy Stroup. That's her actual vocal integrated directly into this video. Everything else is Amy. We did the music, we did the mixing, the mastering, the producing. So what we're doing with, with the vocals is by analyzing the video and knowing the lyrics of the vocals, we're able to match it so we can pull in massive catalogs of vocal content and dynamically add those vocals to videos based on analyzing the video content itself.

This is a hyper personalized world coming at us. It's real wild. Wow.

Yeah, that's a good way to put it. Is. It's exactly that.

Wow.

I feel like going to make an ad now.

I know I'm like, I'm gonna start doing ads. Actually, Telstra took me to Australia and I was hanging out with them, you know, he said, oh, if we're not making our quota at the end of the month, I sit down with a bottle of wine on a Saturday night and start writing Facebook ads. Well, now you just make a video, feed it to Amy, it voices it over, add some music, put it up on Facebook. You got an ad, right? It's like.

And it's fun. It's actually, you know, it really taps into the way people think and I think that's really an exciting part of this.

Well, thank you so much. Anything else that we need to know? Because you've already blown my mind a few times, you know. Got any more in your sleep up your sleeve?

We've got plenty. But I'm hoping you'll interview me again.

Again.

And I get to blow your mind again. So I don't want to give it.

All away anytime you got some mind blowing stuff. I mean, the next 18 months is going to be insane. In a whole lot of levels. Right. I mean, I have 700 companies on my Holodeck list on X. These are companies doing digital twin, you know, virtual spaces, virtual worlds, virtual beings and all sorts of weird shit, you know, and so you're going to see all that come out in the next 18 months. And. And I think this is a core feature of that because we need music in any virtual world. We're going to build or whatever. We're going to build Holodeck, we're going to build.

We've built this service to be on demand and high precision, and that's what's really unique about it. By API, you can essentially generate music that is precisely targeted to your medium at a low cost and in real time.

So I think you want to pitch to developers. Where should developers go to find out more and check out your APIs and check out how to do this stuff.

Everything's on our site. It's Amy fm, Aimi fm right there. And then we. If you want to play around with the sync tool, it's an early release, so we know the UI means love, and we're working on that very, very quickly. But it's sync sync, Amy fm, very cool.

And Ayelet, thanks for bringing me yet another awesome company. I mean, you've brought me so many over the years in Israel and other places. Just real quick, what are you seeing from a PR standpoint? Because this has got to be exciting for PR people because there's. I mean, there's 5,800 companies on my x.com list of AI companies. It's insane how many companies there are right now.

Yeah.

I mean, as you said, we live in an information overload age. And due to that, I think the visuals and videos are so important. Right. And so the fact that any company can go and create a video so simply, it's such a great video as well, is amazing. Right. And it's great for even smaller. Like, the larger companies have the budget to do it, but the smaller companies now can also do it with a small budget. So it's amazing.

Yeah.

What you just showed me is just. All right, well, thank you so much. This has been awesome.

All right.

We're in a new world, people.

Thank you, Robert. I really appreciate your time.

Thank you, Robert. Thank you for everything.