In this end‑of‑week AI news roundup, host Matt Wolfe deliberately shifts focus from the usual big‑lab announcements to a curated set of lesser‑known AI tools and research breakthroughs that have flown under the radar. He begins by introducing Fish Audio’s Drama 3, a 36‑billion‑parameter text‑to‑speech model that lets users embed natural‑language direction tags (e.g., [barely holding it together]) directly in the script, enabling fine‑grained control over emotion, sound effects, and multi‑speaker dialogues. Wolfe highlights its strong pronunciation scores (outperforming ElevenLabs and SpaceX AI) while noting mixed preferences in blind taste tests, the heavy hardware demands (≈46 GB VRAM or 32 GB with 4‑bit quantization), commercial‑use licensing limits, and pricing details ($10 / M characters, rising to $50 after Nov 12). He then moves to ByteDance’s OmniHuman 2.0, an upgrade to the original OmniHuman that can synthesize up to five‑minute continuous talking‑head videos from a single portrait image and an audio track, supporting both speech and singing without seam stitching. Wolfe showcases demo clips and points viewers to the model’s availability. Throughout, he encourages audience feedback on this niche‑focused format and promises more frequent releases if the style resonates.
Chapters
So every week I like to make an end-of-week AI news breakdown video where I share everything that happened in the world of AI for the week. Usually those videos are very focused on some of the bigger labs. They're the ones that get most of the news— OpenAI, Anthropic, Microsoft, Google, Nvidia, companies like that. However, there's often some really cool news that flies under the radar. There's a lot of research going on and smaller tools that not as many people, including myself, are talking about right now. So in this video, I want to share some of the cooler things that I've come across recently that haven't made the bigger end-of-week news videos. Let me know in the comments if you like this style of video, and I'll probably make them more often and share some stuff that's not from the big labs. But let's start here with Fish Audio. Fish Audio released a model called Drama 3, and it's a text-to-speech model where you direct the performance in plain English instead of picking from like a menu of emotional presets. So like most voice tools, they'll give you a dropdown for happy, sad, angry, things like that. Drama 3, however, it lets you write the direction the way you'd like explain it to an actor. You put it in square brackets right in the script. So for example, here's some spoken text with no direction.
I'm fine, really. Go on without me.
And now here's one where there's some direction written in. Barely holding it together, forcing a smile.
I'm fine, really. Go on without me.
And you can even pick specific words. Like right here, they specifically tell it to whisper, go on without me.
I'm fine, really. Go on without me.
So you can like scope an effect to specific words. You could drop in sound effects, things like a small laugh, shaky breath, and you could put them exactly where you want them.
I'm fine, really. Go on without me.
You can even have multiple speakers with multiple voices.
You were there that night.
I was home asleep.
Then why is your coat still wet?
It rained.
They even licensed Uncle Roger's voice for the demo page, which is kind of hilarious.
Hiya, nephew! Why you put ketchup in egg fried rice? Fried rice need soy sauce, need MSG, need wok hei, not tomato sauce. Uncle Roger cannot watch this.
If you've ever watched like any of the Uncle Roger videos, it's actually pretty spot on. If we take a quick peek at artificial analysis here, we can actually see that Fish Audio Drama 3 here scores the highest on pronunciation. So it's really good at pronouncing words, even scoring higher than ElevenLabs and SpaceX AI's text-to-speech model, which is one that I actually demoed in last week's news video. However, when looking at the Arena leaderboard preference, which, you know, sort of did a blind taste test where people listened to a bunch of different models and picked their favorites without knowing what model they were picking, well, it didn't quite do as well. So, you know, it reads hard words better than almost anything else, but a lot of people still prefer other voices. It also appears to be open weight available on Hugging Face. It is a 36 billion parameter model, so you're gonna need some pretty decent hardware in order to run it. People have reported that you need roughly a 46GB VRAM card, or you can get away with 32GB with a community quantitized 4-bit version, but neither of those are gonna run on like most laptops. A couple other quick notes that I found about this one: so the license is free for commercial use, but only if you're making under half a million a year, which is probably a non-issue for most people. The API is about $10 per million characters right now, but that is a launch discount that jumps to $50 after November 12th. And Fish audio takes do vary, so plan on generating a few and keeping the best one. And if you are going to use the API, double-check that you set the model to Drama 3 because it often will quietly fall back to older models and you'll still get billed even if you're using the older models. So make sure you check that. But if you head on over to fish.audio/drama3, click Try Drama 3 for Free, you can play around with it right now. And if you make audiobooks or game dialogue or anything with characters, probably worth trying out. All right, while we're talking about voices, once you have that voice performance, you might want like a face to go with it. Well, ByteDance recently released OmniHuman 2.0. Now this model will take a single image and an audio track, and then it'll generate a video of the person talking or singing or acting it out. Now this is the 2.0 version, and the biggest difference is that the previous version topped out at just 60 seconds. This one will allow you to go up to 5 minutes in one continuous shot, so you don't have to like stitch a bunch of short clips together and hope that you know you don't see the seams or hear the seams. So here's some examples.
The kingdoms called it a legend. Tonight, I make it real.
And it even works with singing.
Now double time, follow the flow. Every expression ready to go. Voicing the feeling, feeling emotion. Wait. Did you
catch that? You can also add in text prompts to control camera moves or add things that weren't in the original photo. Now it generates in 720p resolution, and it doesn't just have to be humans. It can work with like cartoon characters and animals and things like that as well. Now if I'm being totally honest, I still think these lip-syncing models have a long way to go. When I watch a video with like an AI lip sync attached to it, it's still just like isn't working for me. We're getting closer and closer. I mean, like, we're way better than we were a couple years ago, but it still has a lot to be desired. I don't know, what do you think? Let me know in the comments. As far as pricing goes, the price on this one dropped from 12 cents a second in the last model down to 7 cents per second in this model. So if you are making a full 5-minute video, it's still gonna cost you about $21. Now, a few other quick things to note about this one. Everything I'm showing you is ByteDance's own demo And some early viewers did notice like background shifting around in their clip. Again, it's only 720p. You're not going to get 1080p or 4K out of this. And because this is only one photo plus any audio, there's some pretty obvious potential deepfake concerns with stuff like this. Although if you saw what I just saw, I don't think we have to worry about too many people being fooled just yet. This one is available now through Byte+ and through their API. All right, now let's talk about one of my more favorite topics recently. I've been absolutely loving these. Let's talk about world models. So these are AI models that generate a video world that you can move around in. And Tencent's ArcLab released one called World Crafter recently, and this one starts from just a single image or a single line of text. Now, one of the big issues that we've had with world models is that if you're inside a world model and you look away from something and then turn back, a lot of times it's like gone or changed. The model only remembers the last few seconds of what it showed you, and if you move around the room, everything kind of goes wonky. Now World Crafter's fix is a memory that you look up by camera position. So when you point the camera somewhere, it pulls the past views that match where you're looking and feeds those back to the generator. So when you turn back to the original restaurant, it remembers what it saw the first time. So they prove it with this little trick here. They take the video the model generated and rebuild a 3D point cloud from it. If the world were inconsistent, that 3D reconstruction would come out as just like mush. Instead, you get clean geometry and the camera path lines back up when it revisits a spot. It handles moving things too, like a character going out of frame And then later actually coming back into the frame, like we can see with this horse video here where the camera takes the horse out of frame, and then when the camera pivots back around to the right, you can see that the horse is still there even though it was moving. Now, on their own benchmark of 145 different scenes, they say that it beats the other models they tested on revisit consistency and camera control. But just keep in mind that that is Their benchmark. So, you know, take that with a grain of salt. The code for this is available on GitHub right now, and there's even a version on Hugging Face that you can go and play around with right now. In my own testing though, telling it what to generate, it's kind of not the fastest thing in the world, but eventually it gets there. ByteDance also showed off their own version of a world model recently at an event in Beijing. My buddy Tim from Theoretically Media, he posted some footage that somebody sort of took during the event. So it's like this bootlegged camera version. Now, the wildest demo they shared was they took a scene from Stephen Chow's movie King of Comedy and generated a brand new camera path through it. It's like reshooting a shot that was filmed decades ago. You can now get new angles on that shot. You can also type a description to get a world, spawn a giant spoon, and then climb it and walk through a portal from a room into outer space. Now there is a catch, and that's that we can't use it yet. It doesn't even have a public name yet, and there's no release date. And the only footage I've actually seen of it was from people who recorded it at this event, as you know, Tim from Theoretically Media shared here. So yeah, we'll keep an eye on that one, but It looks pretty impressive. Real quick, before we get back to the video, I wanted to show you guys how I've been using this new feature, Dots from OpenAI. Now, I've talked about this in a recent AI news video, and since then I've dug into it even more. Basically, Dots are these always-on AI agents that you can assign ongoing responsibilities to, and they keep on working on them even when you're not using ChatGPT anymore. So for example, I've been using my Dot to help me with my YouTube analytics and updating them automatically every time I post. I connected it to my YouTube channel and it pulls views and engagement data to find patterns across my titles and topics and angles and post times. And I also have a shared space with my team where my Dot keeps all this organized and creates new subpages with weekly reports and video ideas based on what's working. What's cool is that each Dot has its own cloud computer and can use ChatGPT Work and Codex to get things done. It's powered by GPT-6 Astra and learns my preferences so it can proactively suggest things Without me having to ask. So check it out in the link in the description. And thanks so much to OpenAI for sponsoring this portion of today's video. All right. So we already saw 2 world models trying to fix the memory problem. Well, Skyworks Matrix Game 4.0 takes a completely different approach. This one stops asking the AI to remember the world at all. With a normal world model, the AI is doing everything. It's drawing the frame, but it's also the physics engine, the game rules, and the memory. When something leaves the screen, the only record it ever existed is in the pixels the model already drew. And that's kind of why things tend to vanish. Matrix Games splits that job into 2. An ordinary game engine runs the game. It knows where every enemy is, what happens when you shoot something, what's behind you. It draws a cheap, blocky, low-poly version of that scene. Then the video model's only job is to repaint that blocky frame so it looks Finished. So we can see in some of these models down here, on the right is what the game engine actually draws, on the left is what you see. It's the same frame in the same moment, but the engine handles the gameplay and the AI handles the look that's skinned over the gameplay. Since this game engine is keeping track of everything, the world's not gonna forget. A pyramid behind you is still in the engine's map. Whether the AI is drawing it back in or not. Here's another example of a car game. You can see the blocky early rendered version and then AI putting its own graphics over the top. So the game engine only has to remember this and the AI does that. There's also another piece. If you ask for something the game rules don't cover, a coding agent writes new rules into the engine and updates the prompt the video model uses. So the world can change mid-game without the AI having to invent new physics on the fly. Down here we can see 3 games: a medieval castle shooter, an ancient Egypt shooter, and city racing. These long play sessions could run up to about a minute and a half without any sort of cuts. Now there are some catches. Each game has its own model, and each one is about 5 billion parameters each, trained just for that scene. So it's not really a general make me any game model, and the output is 480 by 864, so it's not anywhere near HD. Now the project page does talk about running on a single RTX 5090, but the model card does say there's no guaranteed frame rate, consumer GPU memory fit, or exact scene reconstruction. So it may not totally fit into your GPU's memory, and fast camera moves and long sessions still can cause some drift. And the actual training code isn't included, but you can get the weights and the inference code under Apache 2.0 license, so you can use them commercially. Now there is a Hugging Face space listed here for Hugging Apps Matrix Game 4.0, so you should be able to play around with it here. They do have some examples that are pre-cached in if you want to check out some of those examples. Now personally, I think this engine plus like AI paint job or AI reskin is a pretty realistic path to AI games, at least more so than like asking a model to do everything. Turns out NVIDIA thinks so too. Next up, let's talk about something that came out of NVIDIA called Stream Render H3. This is actually a pretty similar concept to what we just looked at. It uses kind of that same split. Code runs the world and then a video model draws it in real time. The difference with this one is how little the game actually hands the AI. Matrix Game gives the model a blocky 3D render. NVIDIA gives it a color-coded map. Every pixel is a flat color that means road or grass or barrier or rival car. There's no lighting, no textures. The video model actually draws everything you see. So here's the game engine's own view, the color map that it actually sends, and then the final frame that, you know, the user sees. Now, because the map carries no, like, looks at all, you can actually restyle the whole game by swapping one reference image. We can see here that this is all the exact same drive, But when we fast forward a little bit, they can swap out the scene for anywhere they want and put the exact same drive in any different scene. It's the same exact input we see in the middle, just rendered out in different locations and different settings during the day and things like that. So one thing to notice here, this is kind of one of the hard parts for this model, is that the color map paints all of the rival cars the same exact color. So the model has to actually remember on its own which car is which and make sure to keep the same paint job across the various cars. So a fun fact about this model is they actually handed Claude Code the job of building a faster video decoder for this. They gave it a speed target and then they had a working version running in the pipeline in about 3 hours. Now, this does actually require some serious hardware. Each rendering step takes about 218 milliseconds. However, that's on 8 GB200 GPUs. These are like the chips that they put into data centers from NVIDIA. So this isn't something you're going to be running on your gaming PC, at least not for a while. Also, it's only been trained on racing so far. That's why all of the demos are like race cars. So you could put this one kind of next to Matrix Game, which we just looked at, and you get a decent picture of the tradeoff. Matrix Game aims for one consumer GPU at like lower resolution. where Nvidia is going for photoreal, but on a rack of data center hardware. Okay, this one's a little weird, but I wanted to show it off because it's also kind of cool. So Odyssey, the world model company that launched Odyssey 3 not too long ago, posted a clip saying they put a fruit fly brain inside their world model and now it flies around imagined worlds by itself. So this fly brain wasn't actually created by Odyssey. It's a simulation built by a company called Aeon Systems. From a complete map of a fruit fly's brain, about 140,000 neurons and 50 million connections, Odyssey swapped out the virtual body that brain was hooked up to and connected it to a camera moving through an Odyssey 3 world. I don't know, I thought this one was fun, but there's like no paper, no code, no explanation of how the two are connected. And people in the comments did note that the fly never actually tries to land anywhere, which, you know, if it was a real fly, it probably would. But it's brought up a lot of questions about, you know, simulation theory. Like, does this fly believe it's real and in a real world, but it's actually in a simulation? And if so, what are the implications around simulation theory for, you know, humans and stuff? We don't need to go down that rabbit hole though. While you can't actually play with this fly flying around inside of Odyssey 3, you can actually mess around with Odyssey 3 itself. Just head over to experience.odyssey.systems, give it a prompt of what you want it to create, and you can actually interact in this world. I am steering this fox right now. I'm telling it to jump. I'm telling it which direction to go. I'm changing camera directions. All of this is me actually controlling this. Pretty cool. All right, moving into 3D. Preferred Networks, which is based in Japan, just released Telescope, which turns a set of photos of a place into a detailed 3D mesh, like from a single object all the way up to city blocks. Now, the usual way to get a high-quality mesh from photos is to spend a long time fitting a model to that one specific scene. Telescope actually skips all that. It's a generative model, so it's learned what surfaces generally look like, and it fills in gaps where you didn't get many photos. It also handles shiny and glass surfaces, which normally wreck photo-based 3D. So the name is actually the method itself. It builds the whole scene at low detail first, then zooms in on areas that need more detail and refines them like a telescope. The colors here show which level each surface came from. So if we look here, we have Gaussian wrapping, which is sort of the standard method. And if you look at this, it looks pretty good, but it's lacking a lot of detail. Now this is telescope. And there's so much more detail here. So here's the original Gaussian wrapping, and here is Telescope. Like, it's a pretty big difference. Here's another example of Gaussian wrapping, and when I move this over, you can see their version, and all of the details pop into focus. Now, there is a little bit of a catch. You do need a lot of photos to achieve this. You can see this one took 256. This one also took 256 right here. This one actually only took 56, which isn't too bad, and it can only rebuild what's inside the area your camera actually moved around. This is one we don't have any sort of real access to. It does say the code is coming soon, but it looks like a pretty promising leap from what we got out of Gaussian Splatting. Here's another pretty cool one called Gauzy Animate, and this one's actually used for animating 3D characters. It's designed for rigging up characters that are soft on the outside but rigid on the inside. So people in loose clothes, animals, or garments just on their own. So a normal animation skeleton gives you control, but a loose shirt or like a dog's belly doesn't move like bones. So this adds on a whole bunch of other floating nodes on top. These additional floating nodes sort of capture all the jiggle and flow and the additional physics that come along with the, you know, the cloth or the material on top of the bones. So we can see here that it finds this sort of shape But then once it finds the shape, it actually puts a skeleton in the middle to figure out which parts need to actually be rigid. Once it figures out the points that need to be rigid, then it figures out what points need to sort of flow outside of the bones. So here's some other examples, like a clothed human. You have the skeleton, you've got the bones, you've got the skinning, and then everything combined where you can see the, the dress sort of flow. Now you do need a captured motion sequence of the character first. It's not going to just work from like a single image, but the code for this one is available on GitHub and available for research use right now. All right, now this is the last one, and it's actually not AI, but I just thought it was really cool, so I wanted to share it anyway. This one's called World Gacha. I think I'm pronouncing that right. And it's actually a free browser tool from a Japanese developer that actually creates random fictional regions that has mountains and rivers and coasts and towns and Railways and race paddies and almost anything you can imagine, and it draws it as like this manga-style line art. Well, this one's actually not in that manga-style line art. This is a randomly generated city, but you can see if I add it with the lines, you kind of start to get that look. You could also put it on lines and tone and you get like a black and white version. You could put it on this like clay mode, and this is again a randomly generated seed. I could reroll the dice and it's going to build me a whole new town or city or whatever. So now I can zoom in on this city that it just created. We can also change it to downtown. It's going to build us a new city that looks like a downtown area or skyscrapers. It'll build us a new city. And these are again all randomly generated every single time. And you can zoom into different views. So let's say we want like a top view. We can look straight down on this city like this. Maybe want an eye-level view of Central Street, and it actually brings us into the city where we can look around and wander around in this city. Now imagine using something like this and then reskinning it with like one of the AI models we were looking at earlier. You can build a pretty primitive-looking city and then use AI image or video models to reskin it into something really realistic and maybe even build scenes for like films and animation and stuff. Again, this is not actually an AI model. There's no outside data. It's just random numbers and math generating all of this. And if you do like a specific scene, you can actually export it as a PNG or an SVG or even a 3D object file. So you could theoretically create a world and then pull it into Unreal Engine and build a game inside of it or something like that. I know I usually focus on AI stuff, but really, you know, the theme of my channel is just cool stuff that I come across online that I want to share. And sometimes it's not always AI, but that's what I got for you today. Again, I make a news video every single Friday where I break down all of the news that came out in the AI world each week, but it's usually the biggest stories with the most impact on the most amount of people. I'm sort of trying to filter out the signal from the noise of everything that's going on and help you understand what might actually change what you do on a day-to-day basis. And in doing that, sometimes I miss some of these smaller research projects That don't get a ton of love and not a lot of people talk about. And I find all this stuff really cool and really fascinating. And if you go back like 3 years on my channel, this is the type of stuff I used to talk about a lot more frequently until all of the big labs and big companies started having new launches every single week. And I kind of shifted more focus to that. But I want to get back to my roots a little bit and make these videos a little bit more often. So if you like videos like this where I sort of dig and find some more underground things that are happening in the world of like AI and just cool tech research, let me know. I'll make sure to make more videos like this. I'm planning on making at least one of these a month because it's just really fun for me to dig in and find some of this stuff. But again, that's what I got for you. If you like videos like this and you also want to be super looped in and informed on all the latest AI news, I'm going to keep up with all of that for you. I put out a video every Friday with all of the news. I also put out a newsletter twice a week. You can subscribe for free over at futuretools.io. But thanks so much for hanging out with me, nerding out with me today. There's a lot to nerd out on. I really, really appreciate you, and hopefully I'll see you in the next one. Bye-bye.
