Mind-bending AI: The Future Looks Crazy

So I have a habit of collecting really cool AI research and tools that I come across and building it up for weeks and weeks and weeks and then making a video like this to share all of the cool stuff that I've come across over the last several weeks. Some of the stuff is recently released research where we can use demos on places like Hugging Face or Google Collab. And some of it is research that the creators are kind of showing off what it can do, but we don't quite have access to it yet. I love making videos, videos like this because this is literally the future of AI and visual effects. And I love these little sneak peeks into where everything is going. So let's start with image generation. Now, I'm personally in this like AI bubble when I'm on Twitter or X or whatever, and lately I've been seeing a ton of this 0123 plus, which essentially is an AI where you can upload a single image. For example, this fire extinguisher from this single image, it will actually show you what that image would look like from multiple other angles. Here's another example of an image with a ghost eating hamburgers. And you can see that same ghost eating hamburgers from multiple angles and viewpoints. I have this headshot image of myself on my desktop. I'm kind of curious to see what happens when I pull this in. We'll drag and drop it right into this Hugging Face demo here, submit it, and 15 seconds later I have my face from all sorts of different angles. I uploaded this cool AI generated wolf image and I've got different angles on this, but it seems to struggle a little bit with the wider aspect ratio of the image because it kind of cut off the wolf and all the images. But pretty cool that you can upload a single image with one perspective and actually get a different image of all the various perspectives on that image. Here's some more research out of Microsoft called Idea to Image. The code isn't publicly available, but basically it's going to let us do all sorts of cool stuff to generate images closer to what we're looking for. So some of the examples that they share here, they've got this object count. Five people sitting around a table drinking beer, eating buffalo wings. Not bad. I actually think Dall E3 does that pretty well already. A logo suitable for a stylish hotel. And it generated this logo here. Here's another example photo of the object pointed by the blue arrow and a brown corgi. So. So they uploaded an image of an arrow pointing at A ball. And it noticed that this was a ball and it put that same ball with a corgi. This one. They uploaded an image of somebody playing tennis here and gave the prompt a cartoon drawing of Mr. Bean playing tennis with the same clothes and pose as the given image. You can see the result output over here on the far right. Mr. Bean wearing a yellow shirt with the same exact pose as the pose in this picture here. Image manipulation. A drawing with the background change to a beach. So it's this image here, but then they put a beach behind it in pretty much the same pose with the same person in it. They were even able to put two images in here. Photo of Bill Gates with the same clothes as the given image with a dog that looks like this one. In this image, you can see on the far right here, Bill Gates wearing a pretty similar suit and a dog next to Bill Gates that looks like this dog. Just imagine how much more dialed in we can get with our images and get to the exact idea that we have in our mind using some of this stuff. Check this one out. Blending images for new visual design. A logo with a design that naturally blends the two given images as a new logo. The first image is this stethoscope with a paw. And the second image is this little pug image here. And look, it generated this logo of a pug with a stethoscope around it. This one to me is really, really impressive. Again, not something that we have public access to yet. And if you want even more ideas of what this is capable of here, there's all sorts of examples that they share down here that you can click through and see exactly what it's capable of. Really killer stuff. I'll make sure I link to this in the description below. I link to all the research that I'm sharing in this video. And it looks like Matvid Pro also made a video on this exact research. So check that out for an even deeper dive into what this idea to image is capable of. Next up, let's check out Pixart Alpha. Fast training of diffusion Transformers for photorealistic text to image synthesis. Some of these names just roll off the tongue. So first off, just look at some of the images that it generates. Now these are some really good images, right? You've got Real Beautiful Woman Luffy from One Piece. A poster of a mechanical cat. Technical schematics viewed from front. Nature versus Human Nature. In my mind, these are like mid journey DALL E3 level results here. These look really, really good. But we've already got Dall E. And we've already got Mid Journey. So what's so special about this? Well, they've managed to really optimize the training of this model. So if we scroll down here, the training of this is so much more efficient than what else is available out there. You can see the CO2 emissions of the various models that are available. Dall E2 to train it had the CO2 emissions of five humans. Imogen had the CO2 emissions of 0.8 of a human. Stable diffusion 1.5.7. And then Pixar Alpha 0.07. And then the cost to actually train these models. Dall E2, it costs $2.14 million to train. Dall E2 Imagen $366,000. Stable diffusion 1.5 320,000 Pixart Alpha 26,000. So it's just a much more efficient model of training. But the results are on par with something you get out of a mid journey SDXL or Dall E3. I really, really love the contrast in these images. They remind me of like something I'd get out of Mid Journey. It also works with Control. Net, so you can upload extra references, like reference images and the outlines and things like that to dial in the image, just like you would in a normal stable diffusion generation. It also works with Dream Booth, which, if you've watched my past videos, Dream Booth allows you to train your own sort of objects or likeness into a model. So you can actually train your own face into this model if you wanted to, or your own dog's face into this model. And there are some more samples on this page and they're really, really good. Now on their project page they do have a hugging face demo button, but when I click on it, it doesn't appear to be an actual working demo yet. Next up, check out Hyper Human. Hyper Realistic Human Generation with Latent Structural Diffusion. This is a model with one goal in mind to make the most realistic looking humans possible. You can see here a young kid stands before a birthday cake decorated with Captain America. Look how realistic this person is. A man who is sitting in a bus, looking away from the window. You see an image like this on Instagram and somebody tells you it's a real photo, probably going to believe him. An older man is wearing a funny hat in his dining room. Man sitting on brick covered ground, appearing dirty and tired. They've got some other examples down here that compare it to some other models. If we take a peek here, you can see we've got a man riding Skis down the side of a snow covered ski slope. And this is what it generated. Super realistic compared to all these other ones that are here. A pedestrian walk, walks down the snowy street with an umbrella. Ultra realistic. Something going on funky with with his left leg compared to his right leg. But compared to these other images here, pretty dang realistic. The SDXL one's pretty dang good though. And then finally, a man riding on top of a brown horse while wearing a hat. This is their generation compared to these other models. SDXL is pretty decent. But the realism on this model, so dang good. So if you look at this image, the skateboard's all jacked up, but the person on the skateboard looks pretty damn realistic. But compared to these other models, stable diffusion 2.1. Like what's going on? This person's arm is as long as their entire body. Deep Floyd just lacks the detail. SDXL this person's got three legs. Human SD like this person looks like they got ran over by a car. That's just the image generation tech that I wanted to show you. I've got so much more cool stuff to show you here around. Text to video, text to 3D World, text to 3D 3D objects. Before I do, I want to quickly tell you about today's sponsor, which is wirestock. You can learn more about Wirestock over at Wirestock IO, but if you're not familiar with them, they're a platform where you can upload your images or photographs and they will distribute them to all of the stock photo websites for you. And many of the stock photo websites now allow you to sell AI generated images as well. Sites like Imago, Adobe Stock 123RF, Dreams Time, and Freepik all allow AI generated images. So you can generate your AI art, upload it to Wirestock once, it will distribute it to all of these sites for you. Write a description, write a title, add the tags for you and let those sites know that it is AI generated so that you're in compliance with their rules and you don't have to do anything but upload the image. In fact, one other really cool feature of wirestock is you don't even need a tool like Midjourney or Stable Diffusion or Dall E. You can now generate AI art directly inside of wirestock. I click on this generate button at the top and you can generate your own images. You can upload an existing image and reimagine it. You can upload multiple images and mix the images together. And they just added a Brand new feature where you can actually change the face on an image. So if I click into this image, for example, you can see there's now a button down at the bottom that says change face. I can click that click on upload a new face, pull an image of my own headshot in here and now if I click apply it converted the man at the computer to me at the Compute computer. It's a pretty cool new feature that lets you even further dial in the images that you're looking to create before sending them to the stock photo sites. Now, this reface feature is a premium feature, but if you use the coupon code MAT20 at checkout, you get 20% off a premium membership of Wirestock. You can find it all over at Wirestock IO. Once again, if you do decide to upgrade to the premium account, use the coupon code MATT20. Thank you so much to Wirestock for sponsoring this video. I do really appreciate you guys. Here's something I came across on Twitter that I thought was pretty cool from Jared Lu. He says that the latest version of Adobe Express now has the ability to create character animations from your voice. Now he says this is voice to AI character animation, but I couldn't actually confirm that this was using AI, so I'm not 100% sure it is. But regardless, I think it's a pretty dang cool feature. If you check out Adobe Express@adobe.com Express, get Adobe Express free here I can scroll down and under suggested quick actions. There's this one that says Animate from audio. If I click on this, we have the option of multiple characters. And by the way, that character that I just saw a second ago, this guy, he looks very familiar. I think this is a trick that my buddy Olivio Siricas here has known about for a little while because if you go to the end of one of his videos, he's got this little dude. This is the end screen. There is other stuff you can watch like this, but now I know how he did it. Looks like he probably made it with Adobe Express using this toll charact character. But let's go ahead and use a talking taco because everybody knows I love tacos. And I can change the background. I can make it a transparent background and then I can put the animation over any scene that I want. Or I can use one of their existing backgrounds here. I'm a huge baseball fan, so let's put the taco on a baseball diamond. I could scale my character up or down. Let's make it a giant Taco and put it right in the center there. And then I can change the aspect ratio. But I'm just going to go ahead and leave it at 1:1 and I can record. Hey, my name is Matt and the Taco and I love baseball and I love eating tacos. Does that make me a cannibal? Now it says, hang tight, generating a preview. And here's my video. Hey, my name is Matt the Taco and I love baseball and I love eating tacos. Does that make me a cannibal? I think some of the animations are a little bit more pronounced to use one of these actual characters here. Using the taco was probably not the best example to show it off. But if you want to see a really good example, go watch Olivier's videos because he does it at the end of every single one of them. Again, not 100% positive this is using AI, but it's really cool nonetheless and I wanted to share it with you. All right, now let's shift into text to video. Text to video has gotten so good lately. You've got Runway gen 2. You've got pika Labs, you've got Moon Valley, you've got Morph Studio. You've got Animate Diff. There's so many cool text to video options out there. And now we've got this new one called Show 1. Varying pixel and latent diffusion models for text to video generation. And if you look at some of these examples here, they look much more realistic than what we've gotten out of some of the previous text to video. And it even looks like we can generate text inside of our videos now. So check out this comparison using the prompt. A panda, besides, the waterfall is holding a sign that says Show Lab. Here's show one, where you can see the panda, waterfall and the sign that says Show Lab. This one, no sign at all. But you do have a panda and a waterfall. This is Model Scope. Zero Scope. You got some letters in there. You got the panda and the waterfall and then some random letters floating in the sky. And then Gen 2 is. Well, you can see what I see. Here's some more examples. Look at the snail right here. Snail slowly creeping along. Super macro, close up, high resolution, best quality. Model Scope's actually pretty solid, but I mean, look at the colors in show one. It just looks so good. Zero Scope, we've got like a swarm of snails and then Gen 2. Not great compared to the rest. Speaking of text to video, we have new research called Motion Director Motion Customization of Text to Video Diffusion models. Now, this is really cool because you can actually give it an input video and then have it generate new ideas based on the combination of the input video and your text prompt. So they uploaded a few videos of people lifting weights and then gave it the text prompt. A bear is lifting weights. And you can see. Here's a video of a bear lifting weights. Sort of following the way the video was doing it. A dog is lifting weights. You can see. Here's the video of that. They input a video of like a drone shot going around this house here. And then they gave a prompt of a pyramid in a forest. And you can see it sort of circles around this AI generated pyramid in the same way it circles around the house. A temple on a mountain. Same idea. Here's a input video of a car running on a road. They changed it to a tank in the desert and a tiger in the forest. And it sort of followed the same path as that car. Here's some other examples here. Some input videos of people playing golf. You can see it compared to tuna video and several zero scope models. And then the motion director model, which looks a lot more like a monkey playing golf than the rest of them. And this page has several other examples that you can explore and see what this model is really capable of. Now let's talk about audio with AI. This research recently came out called Salmon Speech Audio Language Music Open Neural Network. It allows speech, audio and music inputs. And you can essentially chat with the audio to ask questions about the audio. And this one does have a working demo live over on Gradio here, for example, you can upload an audio. They have some examples here. Let's go ahead and use this first Gunshots one and take a listen. Can you guess where I am right now? So if I click upload and start chat here, I can ask it a question. What sounds are heard in the background? Gunshots and explosions are heard in the background. What is the person saying? The person is saying, can you guess where I am right now? So it listens and it can hear both the sound effects and the person speaking and understand it. Where is the person from? The audio person is from the United States. I was looking for like war zone or something, something like that, but you get the idea. You can upload audio and then ask questions about the audio. Here's something really interesting that's been circulating around the web lately. It's this view of a car on fire. Now, the environment is the real world. So everything you see, the cars, the streets, everything in the background, this is actually real world, but the car itself is actually computer generated. So the car, the fire, the smoke, this is all computer generated imagery. You can see as they move this little pill shaped thing through it, the fire and the smoke is all impacted by it. It's just so crazy because this is the type of thing that makes it harder and harder for people to actually believe their eyes when they see videos on the Internet. Now do keep in mind, this video has been circulating on Twitter and somebody actually wrote, this video was generated in Unreal Engine. It's crucial to understand what fifth generation warfare looks like. Social engineering and misinformation is the name of the game. But the creator of this actually said, my work is being reposted on Twitter as misinformation by claiming that it is misinformation. So do your own research. They then wanted to say, to give more context, it is in fact not generated in Unreal Engine and not rendered in real time. But the point still does stand that seeing stuff like this does make it pretty hard to believe our eyes. Now, once again, I don't necessarily know if this has anything to do with AI. I don't know if any of this was actually AI generated. I don't believe it was. I just thought it was really cool and wanted to share it with you. Now check this out. This is called 3D GPT procedural 3D modeling with large language models. And this is a text to 3D scene generator. So here's an example. The desert, an endless sea of shifting sand stretched to the horizon, is rippling dunes catching the golden rays of the setting sun, creating an ever changing landscape of shadows and light. Or the lake, serene and glassy, mirrored the cloudless sky above, reflecting the surrounding mountains. And the graceful flight of a heron as lily pads floated like emerald jewels upon its tranquil surface. And from that prompt, it generated this 3D scene. Blinding sunlight reigns over the vast desert expanse, casting sharp shadows behind the few resilient trees. Small sand piles sculpted by the relentless wind pepper the golden terrain. If you look at it, you basically enter a prompt, it creates the scene, converts it into Python code, and then the Python code then becomes a 3D model in Blender. So these scenes that you're creating, you should be able to pull into Blender, Unreal Engine Unity, any of those tools, and use these 3D scenes in your games or your video creations or whatever you need 3D scenes for. Speaking of 3D scenes, we also got this research called Dream Space. Dreaming your room space with text driven panoramic texture propagation. So with this, you can actually film a real world scene. You know, walk around with a camera and film and it looks like it's creating a Nerf or a Gaussian splat or that sort of thing. And then reconstructs the scene into 3D objects. And then you can apply text prompts to change what that room looks like. So you've got Sci Fi theme here or Zelda theme here where it looks like you've got Hyrule out the window there. But what's even cool is then they show off. You can actually then look around in this room in virtual reality. It looks like they're using a meta quest or something here to look around this new room that they created with the prompt seeing through the galaxy. Here's another example of here of somebody's like apartment or house or something. You can see the 3D object scene that it created. Applied the prompt cyberpunk, the prompt Nebula, the prompt anime landscape and the prompt Harry Potter and got got completely different scenes out of each one here. So next let's look at any portrait gan animatable 3D portrait generation from 2D image collections. So this is research where you can actually upload 2D images and then actually turn them into sort of 3D movable avatars where even their lips move and they talk. And you can see them smiling and opening their mouths and things like that. And I actually just find this animation here borderline hypnotic. I hate to admit it, but I stared at this for way longer than I care to share with you. But these are examples of these characters that are animated. You can see moving their head, they're smiling, they're looking around. And the characters can actually be driven by real video. So you can see a video of a real person talking here. And then these animations are actually following the real person talking. Same with this video below. Somebody talking. The animations follow along. Now let's talk about text to 3D, because this is something that has a massive leaps and bounds recently. This one's called GSGEN text to 3D using Gaussian splatting. You can see all sorts of examples here where they were able to create these 3D objects using text prompts and Gaussian splatting, which ever since Gajan splatting came out, I don't know, four or six weeks ago, it's created these massive leaps of what we can create with 3D scenes and 3D objects. We've got a plate of delicious tacos, a car made of sushi, a furry corgi, a pineapple and all of these are pretty dang good looking. Here we can see it compared to previous models. Look at this Corgi in Dream Fusion compared to their version. This Panda in Dream Fusion compared to their version. Obviously it's quite a bit better. If you want to know how it actually works and you know how to interpret this, here's a screenshot that shows how this works. It's a little bit over my head, so it's not something I can totally explain how it works, but. But from my understanding, you give it a text prompt, it generates a 2D image, then tries to generate a 3D point cloud from that 2D image, and then with that point cloud uses Gaussian splatting to turn it into a 3D image. But I don't totally know what I'm talking about, honestly. And then we have Gaussian Dreamer. Fast generation from text to 3D Gaussian splatting with point cloud priors. This sounds like it's using a very similar method to what we just looked at, but they look even more detailed. And this one, I feel is a little bit easier to understand. We take a look at this image here. The prompt was given a fox. It uses a 3D diffusion model. It generates a point cloud. If you remember point E, that was one of the earlier models that was doing text to 3D and they didn't look very good. They created these sort of like point clouds of the 3D object, but they weren't very detailed. It then takes this point cloud and using Gaussian splatting, fills in the details of the points and gets a much more clear, slightly more realistic image. Here's an example of an axe where it starts with this point cloud and then I believe using Gaussian splatting, it sort of filled in the details there. Here's some other examples here. An airplane, a dragon, a flamethrower, a magic dagger, a mushroom boss, a fox, a banana, a jellyfish. Lots of cool examples. And the detail of these 3D generations has just gotten so good compared to what we just had weeks ago. It's mind blowing, honestly. Sticking with the theme of 3D generation, this one's not using Gaussian splatting, I think, I don't think. But it's called MV Dream Multi View Diffusion for 3D generation. And here we can see an example of what the process looks like as these images are generated in 3D and it generates both the object, but it also, on this version, puts a texture over the object. Here's some other examples here there's just the object untextured and then obviously with the texture applied to it. Here's just a object of Gandalf. And then if I apply the texture, you get that. And here's some comparisons against previous models. Dream Fusion Magic 3D text to mesh Prolific Dreamer and then our model using the prompt an astronaut riding a horse. And you can just see how far these text to 3D models have really come just in a short amount of time, honestly. And what I find really cool about this is you can actually train your own images into it using Dream Booth. So here in this example, they trained images of a very specific dog into it and were able to generate 3D objects in different positions of the dog they trained into it. So here's one of the dog sitting, one of the dog jumping, one of the dog on a rainbow carpet, one of the dog sleeping. And these were all generated from a trained in image of their dog. So text to 3D. Blowing my mind right now how far it's come. Once again, one of my sort of longer term goals is to try to create a game using something like Unreal Engine. And now we're getting text to scene generation with that 3D GPT and we're getting really, really good 3D objects with text to 3D. And I'm really excited because creat creating game assets is going to be a lot easier with these tools. Creating game environments is going to get a lot easier with these AI tools. I'm really excited with the pace at which all of this is moving. Obviously, if you've been following along to my YouTube channel, you're seeing the pace that all of this is accelerating at and it's just getting so exciting and so fun. Anyway, that's all I got for you today. I love making videos like this, where I break down all of the advancements as they're happening and try my best to explain them, even though I often don't even know what I'm talking about myself. But it's fun to explore and see where things are heading and look at where the future of AI is going. If you're just watching the news videos and just seeing what's available now, you're only getting half the picture. With videos like this, you get to see where things are going next and get ahead of the curve. And I love being ahead of the curve on this AI stuff and hopefully you do too. So thank you so much for nerding out with me. This was a total nerd fest for me and I'm excited to make more videos like this for you you haven't already? Check out FutureTools IO. This is where I curate all the cool AI tools that I come all the latest AI news on a daily basis here and I've got a free newsletter. You can click this button, join the free newsletter and I will send you all of the coolest AI tools and all the latest AI news directly to your inbox. You can find it all over at FutureTools I oh so thank you so much for tuning in again and thank you to Wirestock for sponsoring this video. I really appreciate you guys and if you enjoyed this video maybe consider liking it. And if you want to see more videos like this I'd love it if you'd subscribe to this channel. Subscribe. It would really make me happy. So thank you so much. Really appreciate you again. I can't say it enough. I'll see you guys in the next video. Bye bye.