Amazing Free AI Composer: ACE-Step Now Available

Hey guys, welcome back to the Matvid Pro AI YouTube channel. We've got access to a brand new Apache 2.0 licensed meaning fully open source AI music generator. It's called Ace Dash Step and it was just released by step fun AI and Ace Studio in combination 3 1/2 billion parameter open weight model, Apache 2.0 license. It even supports lyric generation here and it can generate four minute long songs in just 20 seconds. But that is on an A100 GPU so that is definitely not consumer level. We'll talk more about running it locally later in the video, but for now I want you guys to take a listen to what it sounds like. I think you guys will be surprised. This is the closest open source variant we have to compete with Suno AI Udio AI or Refusion. It's not just a dream it's the.

Start of a way Building the future where music will play A modal that listens A modal that grows A step is the rhythm the new world knows.

Grinning our eyes Winding our hair Breaking.

Rules we don't even care Shouting loud.

Above the crowd Living life like we're all about.

So not bad, right guys? It's pretty crazy. This is a really well developed project. There's a lot of work put into this and they've set us up, essentially set the community up to really get down to work and start playing around with these new models, building loras and fine tuning them in different ways. Because it is Apache 2.0 fully open source, but they did build two Lora adapters, lyrics to vocal and text to sample on top of a step to showcase how subtask fine tuning can potentially fit into music creation workflows and kind of a little bit of a demonstration. Not bad for open source man.

Sack, saxophone foreign.

I'm impressed. I mean this is definitely one of the larger AI music open source model releases I've seen in my day, if not the largest and most ready for us to really experiment and play around with. So yeah, this thing does have a hugging face Spaces free page demo where you can try it out. I'm definitely going to be messing around with it in here today, but you could run it locally if you have a beefy GPU at some point. I would love to see the community try to distill this down somehow into running on less vram. I'm sure it's definitely possible to some degree. I just don't know what the sacrifices are going to end up being. All right, taking a look at the readme on the GitHub itself. For local performance, you're going to need something beefy. They did manage to get it working on a MacBook M2 Max, which is very interesting, but it runs a lot slower than the Nvidia GPU. GPUs all the way down to a 3090 though. If you guys have a 3090, you could actually run this locally. It seems like the VRAM requirements are going to be up there though. Looks like 20 plus gigabytes. At a lower quality or a lower step amount, 3090 is generating a minute of audio in about 5 seconds. Nvidia A100 business grade GPU is a little over 2 seconds and the RTX 4090 is under 2 seconds to get a minute of audio, and that is at the lower settings. Step quality. If you want to bump the quality up to the max 60 steps, you're looking at about doubling your time for each. But still, even on the 3090, 10 seconds for a minute of audio is not bad at all. At the max quality for a full length song, you're looking at maybe 30 seconds on the lower end hardware that can support this, at least in consumer terms. But this is definitely not a one click install, at least not yet. I hope to see this on something like Pinocchio in the future. So Cocktail Peanut, get on that man. We would love that. But yeah, there's. There's kind of a lot of steps to actually getting this thing running locally if you do have the beefy hardware to do so. I do want to kind of go over the GitHub a little bit more, get into the nitty gritty. It supports all mainstream music styles, apparently with various description formats including short tags, descriptive text or use case scenarios. Supports up to 19 languages. Fantastic to see top 10 well performing languages including English, Chinese, Russian, Spanish, Japanese, German, French, Portuguese, Italian and Korean. Less common languages may underperform though. It's capable of producing realistic instrumental tracks as you heard in the intro, complex arrangements as well with multiple instruments while maintaining the musical coherence. We'll put that to the test, but it did sound good of course. Also capable of various vocal styles and techniques with decent quality. It does seem to have a level of audio inpainting as well, meaning you can upload audio, take a chunk out of it and then have this model regenerate or hallucinate whatever the audio in between would be. Could be very interesting for various use cases. Maybe you upload like 15 seconds of a beat and you want it to transition into something else. But the AI holds on to that initial 15 seconds of beat throughout the whole song. Apparently this thing also supports something called lyric editing, which I haven't really seen before. Flowchart edit technology to enable local lyric modifications while preserving melody, vocals, etc. Works great both with generated content and uploaded audio, greatly enhancing the creative possibilities here. I really would like to see more about that. Okay. And then they go to the Loras that we saw demos of in their little video. And some things coming soon. A rap machine fine tuned on pure rap data to create an AI system specialized in rap generation that'll be fun to see. Expect capabilities include AI rap battles and narrative expression through rap. And the reasoning behind this is that rap has exceptional storytelling and expressive capabilities, offering extraordinary application potential. Interesting. Interesting. Okay. Stem Gem a control net Laura trained on multi track data to generate individual instrument stems. Very intriguing. You know what? This is something that we have seen on some other AI platforms before. So glad to see this is included because I've heard from a lot of people who are actually in the music field that their main issue a lot of times with these AI music generations is that they don't have individual instrument stems that you can adjust, tweak or change once your generation has already completed. But I believe some generators have implemented variations of this. But it's great to see this coming soon on a fully open source AI music generator. Also, singing to accompaniment. The reverse process of Stem Gem generating a mixed master track from a single vocal track. So essentially you would just drop the raw singing in the raw vocal track. And you could even use probably your own raw vocal track if you recorded one yourself. And it would create a full instrumental backing that complements the input vocals, making it easy to add professional sounding accompaniment to any vocal recording. Very cool. So I'm really, really excited actually for all of these upcoming generations and tweaks to this. These seem like they're gonna be super fun to play with, but let's actually mess around with this thing, test it out for ourselves. Let's start off with like a 37 second generation. All right, whip up some quick tagged song lyrics about the McDonald's ice cream machine being broken. Go ahead, chat GPT broken dreams and Sunday screams. Hashtag Mcbroken. So sad. All right, Amazing. Okay, this should be enough to kind of get us going. We'll paste those lyrics on. And you know what, I'll keep the bass tags here. Rock, melodic guitar, drums, bass, keyboard. They've also got some basic settings, the step length, so 27. This means we're on the lower end of the scale here for medium end of the scale for generation quality. They've got guidance scale, of course as well, which I'm not going to touch since I've never used this model before. Manual seeds, of course, and then advanced settings so you could change the scheduler. I see. CFG type. Okay, I see. I'm again just going to kind of leave everything on default for now. I don't want to mess with stuff. Let's generate. Oh, the queue is quite long eight out of eight. Oh, seven out of seven. We're moving forward even with a long cue. 100 seconds. That's not too bad for a free hugging face space. And over here you can see there's all different other things that you can change. If you want to do the repainting instead, the editing or the extending. I'll be really interested to see if the model tries to fit all of these lyrics into just 37 seconds or if it's just going to generate like half of them, because I imagine this is longer than 37 seconds to actually sing out. All right, our generated track is here. Let's take a listen.

Pulled up smooth sun hitting the ride Craving that cone with the soft serve vibe Speak of crackles Can I take your order?

Yeah.

Give me that twist Cold drip, no border the machine ain't down, man man why this always going down in my town? Golden arch has got me feeling betrayed Like a Sunday dream that you swole get made.

Wow. Not bad. Okay, so it didn't try to fit everything in. We kind of just got like the verse and the hook, but it actually sounded pretty good, especially for being on the mid level quality. If we go to basic settings, if we up these infer steps, we'll get a higher quality. I think the voice, yeah, it does sound a little bit AI generated. Doesn't sound as good as the closed source models we're used to. But this is open source, this is free to use, and we've got plenty of different parameters to mess around to tweak things and kind of try to get exactly what we want, which oftentimes you won't get in something like sunoai. You got to keep in mind, even with these open source projects that don't run well on most consumer GPUs because they don't have enough VRAM, there's still a lot of benefits. Even if we're using it online through hugging face. The ability to adjust all of these settings separately, the ability to use it for free, and the ability for very smart Community members to modify and change everything to maybe possibly make it work on lower end hardware or make it even better. Right. There's a lot of potential here, but let's actually up the steps here. I want higher quality. Let's try 40. Make a longer song here, 120 seconds. Probably edit a few of these lyrics as well. We don't need the McSad frozen hard and McFlurry denied in the middle of the lyrics. And just scrolling through over on the side here, I see how it works. So you have the repainting start time and the end time to kind of specify exactly the point in your song you want to repaint out and have it regenerate and then also a variance again, all of these different sliders, all these different options to tweak things and kind of make it sound exactly as you want. It might take a long time, but you could get, you know, the perfect sounding song just by using this open source model and spending a lot of time tweaking settings such as variants, for example. Not exactly sure what this would do, but again, we've also got the edit tags here. Edit lyrics. You can upload a song and then change certain parts of it. Apparently this is not something I'm used to with the traditional music generators that are closed source that we've had for a little bit now in the AI space and then extension extensions. Pretty obvious on how that works, but yeah, I love to see all the options in here. It really is a fully fledged project. They did a great job getting this into our hands as something that is ready to go. All right, let's see how this goes. Definitely. Already sounds better. The extra steps make a difference.

Pulled up smooth sun hitting the ride Craving that cone with the soft serve vibe speaker Crapples, can I take your order yet? Give me that twist Cold drip, no.

Border the machine's down again Watch this.

Always going down in my town golden arches got me feeling betrayed Like a Sunday dream that just won't get made Midnight cravings got me back on the strip Drive through hopes with that vanilla whip but it's maintenance mode like it's part of the script that McFlurry machine got me feeling it Clips Why even.

Show it on the screen if it ain't in play like promising love then.

Ghost in the day I ain't mad oh, Cam, you just robbed a grown man of his dance cream plan.

It's struggling a little bit with some of these. Definitely trying to hit the full time. We'll get to the outro. Okay. It did pretty good. It mostly followed the lyrics. Obviously guys, this isn't going to be the quality level that we expect from Suno v4.5 or Udio. It's open source. It's a little bit on the lower end, especially when you compare to these companies that have been around for a while. But it is the most impressive open source music generation I have heard by far and the most customizable. It got most of these lyrics and it got them mostly down. A couple of screw ups, maybe here and there, a couple of lyrics read incorrectly or pronounced incorrectly. But once we got to the end here, it kind of did like this guitar solo outro thing and then it didn't actually finish up all of the lyrics. It probably would if we gave it more time to generate, but I think it definitely had time with that insane guitar solo to actually generate the outro text. So yeah, I mean these models are all gonna have these different quirks, different things, different parameters. And there is absolutely ways to adjust all of these settings, including these crazy advanced ones to perfect your output. But it might take some serious time. Alright, let's try some deep electronic trap rap hip hop synth pads. Typically pretty difficult for music generators. I'm gonna quality even more to 50 now. All right, now I need a rap about how the spring pollen has made my eyes swell up like watermelons. Chat GPT give me the goods. Why is it so hyped? Let's go MC sneeze in the puffy squad Watermelon eyes my word Chat GPT again, do not need these random hashtags. Hashtag allergy beets or Benadryl blues Delete those puffy eyed problems. Flower War 2025 Again, this is a little bit long. Generate. Whoops. Oh my God. I keep clicking the sample button instead of the generate button at the bottom. Very aggravating. Oh, I'm excited. I think this one's going to be a good one. I can feel it. Alrighty. Allergy wrap has finished. We accidentally generated like three and a half minutes of audio. Whoops.

My face looking like a picture with me let's go, let's go. Woke up feeling cute Mirror said you lying My eyes puffed up like fruit that's been frying Springtime breeze got me wheezing in bed Nose doing more runs than Olympic legs Eyes better than a demon in a breakout arc look like a box 12 bees in the dog park Doctor said clarity and I said too late fam I'm already looking like in and play jam I got watermelon eyes can't, can't see the skies Got pollen forecast and a farm supply if you see me out outside toss me so crag Cuz I'm crying like a toddler with a.

It started so good Then they've been.

Looking smug in its yellow.

Okay, I mean, this makes sense that it's not great with the. The hip hop and the rap. The beat is definitely there. It's just the. The lyrics, you know, actually rapping them out. They were working on a specific fine tune to help out with this, and it kind of makes sense. Why? Because it's not great at rap life.

So salute to the real ones with the spray time curse who blink twice a minute still feel worse.

Yeah, I mean, this guy sounds like he smoked 80 packs a day, man. Something ain't right here. All right, convert this to a country ballad. Typically, these AI Music Gen models are pretty good at country music, so I think this will go quite well. Watermelon Eyes, Spring Ain't Sweet by Dusty McSniff and the Claritins. You gotta love some of the. The chatgpt humor. Actually kind of funny. All right, let's paste those lyrics in. You know what, chatgpt, you can make the tags too. Give me a list of tags for the sound of this song. No, chatgpt, that is. This is not what I wanted. I didn't need a bunch of hashtags. I mean, how many tags can we fit in here? Okay, well, these lyrical themes like allergy tags and stuff, we probably don't need those. Yeah, or rural. Slow to mid tempo. Okay, we'll just keep it at this. We'll keep it kind localized. Let's max out the inference steps generate. And there is our song at three minutes.

Woke up this morning felt something wrong cuz Couldn't hum a song Mirror see the sunrise Swollen red moons my eyes blowing I caught air balloons. But them flowers don't smile they just blow me away I feel the daisies ain't beauty to me it's just a battleground for my ology.

It's having trouble with the pronunciation and stuff. Let's keep the guidance scale at 15. I suppose I'm trying not to adjust any of the settings here. Maybe it's all the tags. Maybe we have too many of them. All right, I'm just gonna do one tag. Country ballot, leave all these settings on default, see how it goes. Let us take a listen.

Woke up this morning felt something wrong Couldn't see the sunrise Couldn't hum a song Mira showed me two swollen red Moons my eyes blowing like hot air balloons Spring done rolled in with a yellow bouquet but them flowers don't smile they just blow me away A field of daisies ain't beauty to me it's just a battleground for my ology I got waterloo eyes and a nose that won't quit sneezing like thunder can't handle this April done curse me with her sweet little eye Sick go outside now I can't even cry oh this ain't the spring I know new It's a hate fever Heartbreak with a snot filled blues My girl went picking wild flowers near the stream I stayed in the truck with a Z dream she's dancing in pollen like a carefree breeze While I'm over here wrestling with birchwood disease Doctor said, son, it's just how it goes when your immune system rides country pro so I strum this guitar got no alibi Said I'm in love with a season that makes me cry Got water bright regrets Allergic to life and romance and pants. Grass now it just reminds me of wiping my past Lord give me strength for some seedless relief Cause these ologies cut like a cowboy's dream.

It's. It's good. It's. Having a tough time with the pronunciation.

She's a stinging siren in a flow dress and I'm just a cowboy with a box of tissues.

Is it gonna see again? Kind of did a decent job with that last delivery. It's really a mixed bag, man. With this again, like, you're. This is one of those models where it's like you're gonna. You're gonna have to spend a lot of time going through, seeing what works, seeing what doesn't. I would suggest starting with only maybe, you know, the first verse and chorus and spending some time trying to fine tune in the right sound with all of these settings, you know, the different inference steps. It seems like upping the inference steps might actually be introducing a little bit of weirdness with the pronunciation of certain lyrics again. But there's like guidance scale. Guidance scale for the text, all the different scheduler settings. And let's say you did get a very close to perfect generation. You could put it in and do some of the repainting or the editing or even doing extensions on, you know, smaller parts. Like, start out with the verse and chorus and then do an extension and kind of builds your song up from there, similar to how you would in Udio. It's definitely possible for something. Apache 2.0. Fully open source. Out of the Gate it's set up super well. Easily the best open source music generation model I've ever heard. Not exactly on the level of Udio Suno Refusion, but it's good enough for a start and you could produce something that is genuinely listenable with this, given enough time to mess around with it and get your perfect generation out the door. I'm super excited to see what the community actually does with this. It's out there. I mean, if you're a dev, if you're an AI enthusiast of sorts, I recommend checking this one out for sure. Especially if you have a big honking GPU at home to run it locally. Seems like it generates pretty quickly. Let me know what you guys think in the comments down below. I know a lot of you are probably going to be disappointed that this doesn't work on a lot of lower end GPUs, but I say open source and especially at this level of quality. Providing all of these additional tools, the repainting, the editing, these different dials we can tweak. You can't ask for too much more than that. Sometimes it's got the quality level to back up the amount of GPU you need to run this for. Sure. Thanks so much for watching folks. I'll see you in the next video and I can't wait to hear more on this specific model. See you in the next one. Goodbye.