CRAZY AI Computer Graphic Technology!

Today I want to talk about some of the really cool advancements that have been happening in the world of computer vision, computer graphics and visual arts using AI. There's been some really cool advancements and really awesome stuff that I've been seeing this week that gets me really excited. So I want to show it off to you. So today, April 5th, the day I'm recording this meta introduced Segment anything. This is a tool where you can feed it an image and it will actually segment out everything inside of that image. You, you can also feed it a video and it will segment out all of the things in that video. Even as the camera moves, you can see that it still maintains and figures out what thing it's supposed to be focused on. Now this has some huge implications in the world of AR&VR and that it will be able to actually track where the eye is looking and then focus the eye in on exactly the thing that you're looking at. It's also gonna be amazing for photo editors because it's gonna make it a heck of a lot easier because to isolate certain things in the image and hide either that thing or hide whatever else is behind that thing. Here's some examples of it being used. Here's a picture of a frog. And as they move the mouse around, you can see that they can isolate certain things on the frog and it happens in real time as they hover over the thing. It can also automatically segment everything in an image. If you look at this picture here of New York City, it automatically outlined all of the buildings, the bridge, the boat, the water, everything. It segmented it all out. It it can also generate multiple masks of an object. You can see it segmented out this mannequin shirt and jacket as separate objects. It says here SAM can take input prompts from other systems, such as in the future, taking a user's gaze from an AR or VR headset and select an object. Now, yes, this kind of technology and computer vision has existed for a while, but with this new meta version called SAM or Segment anything, it sort of makes it available to everybody to use and they open sourced it. So anybody can go and build off of this and iterate off of it if they'd like to. I could imagine this being used in all sorts of new workflows. Imagine AI generated art that then gets fed through this to look at the image to explain what's in the image and then remove certain items. This will be built into various workflows now that it is widely available to anybody that wants to use it. And they even have a demo on their website that you can play with yourself if you want. They have a bunch of example images already loaded in that you can play around with. For example, if I click on these people playing cricket here and I move my mouse around, you can see that in real, it finds the objects in the image. If I want to select just one player, I can hover over that player, click on them, and it will outline that player. And then if I come over here, I can click cut out object, and you can see that it isolated just that player. Now, if I go and open this in a new tab here, look, I have an image of just that player cut out. I did that all in real time. Right now they've also got a box feature where you can draw boxes around certain things. So let's draw a box around this player. You can see that it found just that player in that area. If I draw a box around this player here, it found just that player. I can draw a box around this one. Even if it overlaps with this one a little bit, it knows that I'm just trying to highlight this player here. There's also this option to click everything and it will scan the whole image and try to break out every element of that image. You can see it's got the field, it's got each individual player. It's even broken out the pants and the shirts from the individual players. And if I want, I can click cut out all objects. And you can see that it generated cutouts of every single object here. And I could pull these into Photoshop now and actually save them as layers and have every single one of these elements be its own layer inside of Photoshop if I wanted to. All right, let's try it with our own image. Real quick. Let's click upload and let's pull in an image that I generated with AI here. Here's an AI generated landscape photo that I created. There's a lot going on in this picture, so I'm really curious to see how it segments it all. Let's go ahead and have it segment everything and see if it finds all of the individual elements. It drew an outline around pretty much everything. It found this tree, this tree, this mountain, these trees. It sort of separated out everything real nicely. Let's try the hover and click here. If I hover over this tree, you can see it's isolating just that one tree there. If I hover over just this tree, it kind of thinks that this upper tree is part of this tree. So if I Go remove area. I could probably click up in this upper part. And there you go. It determined this part up here was not actually part of the tree and isolated, just that I could click cutout object and it'll give me a cutout of just my tree. And once again, I can open it in a new tab and get an okay outline of just that tree. Let's try another image real quick. Let me grab this image of these two boxers here. Let's see how it does. All right, so it's kind of finding each area individually, but if I hover it over just the right spot, you can see it actually finds the entire boxer properly. Even notices that the rope is cutting through the boxer right here. So if I was to click cut out object, you can see here's what it cut out. You could actually see where the rope went across as well. Let's have it cut out this other boxer for us here. And there we go. Now we have this whole boxer. If I click cutout object now I've got a cutout of just this boxer here. Now what's this? Multi mask. If I turn on multi mask, Sam predicts multiple mask possibilities with a single click. Select an object to start. Let's go ahead and click on this boxer again. And look at that. It broke out his pecs and his body separately and it does a cool animation when I move my mouse over it. All right, let's go ahead and reset it and let's do it with this boxer now. So it found these separate layers for this boxer. Go ahead and let it scan everything, see what it finds. And that's how it separated the layers when I do scan everything. So pretty cool stuff. Again, this really democratizes computer vision technology for pretty much anybody. I'm excited to see how the people that have access to the open source version of this iterate off of it. How it gets built into other existing workflows. I can see other existing AI image generators building this into their workflow so that you can easily remove elements that you don't want from the image. There's going to be some really cool stuff that comes from this seemingly simple technology, but it's actually really, really impressive when you see how quickly it works and. And how simple it is to use for anybody. All right, so I want to break down some other really cool stuff that's sort of in the world of visual effects. And this has implications in video games. This has implications in creating really cool cinematic, computer generated movies and videos. It has implications in virtual tourism and things like that. So if you're not familiar with Unreal Engine, Unreal Engine is probably the best game engine out there that some of the most realistic looking video games are developed on top of. And if you're not familiar with Luma AI, they're one of the leading companies behind a technology called NURSE or Neural Radiance Fields. And a NERF is essentially, you can take a camera and move the camera around an object and look at that object with the camera and it will generate a 3D version of that image. You can also take your camera and move it around an environment and create a 3D version of that environment. Well, the cool news that came out this week is that Luma AI is and Unreal Engine have sort of teamed up and now there's a plugin where you can use these Nerfs with Unreal Engine. And here's the video that they released to show off this technology.

So where this robot is sitting down, that was all generated with a nerf. This world, this environment that the camera is scanning through was created with NERFs. This motorcycle was created with a Nerf. Somebody walked around that motorcycle with a camera. This environment that these things are sitting in. So somebody walked through this environment with a camera and generated this. So this is a actual real life place that somebody just filmed with their camera and turned into a 3D environment. So what does that mean? That means that creating 3D worlds just became accessible to literally anyone. If you have an iPhone or really any smartphone, you could walk around your own house with your camera and just film the video of what your house looks like and then run it through Luma Labs Nerf creator tool, and it will create that house, that world that you walked around in, in a 3D environment. And then you can import that into Unreal Engine and walk around in a game. And the level, the world you're walking around in will be your actual house that you just filmed. How cool is that? This is going to democratize video game development, video creation. Anybody can create 3D environments now by just walking around with a phone and filming that environment and then importing that environment into Unreal Engine. And now you can walk around in that 3D environment inside of Unreal Engine. That is pretty fricking unreal. And by the way, Unreal Engine is a free tool to use. Anybody can create on it. If you go over to Docs, lumalabs, AI, slash that URL there, you can get instructions on how to set up Luma Labs inside of Unreal Engine. Now speaking of generating 3D worlds. There's another company called C Skybox Labs and this is a text to 3D world generator. I can actually move my mouse around and look at this 3D world here. And I can generate any 3D world I can imagine. And theoretically, just like with the Nerfs and Luma Labs, you should be able to import these giant 360 images into something like Unreal Engine and use them as sort of the background, the backdrop of your video games. So if I wanted to type something like a colorful Minecraft village with lots of trees and then for the landscape, let's do modern computer animation and generate it and let's see what we get. And look at this. It created a Minecraft looking 3D world with trees that I can look around in and get a 360 degree view of. And if I want, I can download this whole thing right here. You'll see if I open this up, it's just a giant wide image that I can use as my backdrop in something like Unreal Engine if I wanted to. A medieval dark world lit by torches filled with dragons. Let's see what happens. Let's do fantasy landscape and let's generate. All right, here's our medieval dark world lit by torches. Is it filled with dragons? I'm not seeing the dragons, but you can tell it's a world that would have dragons. And it's still pretty damn impressive looking. Like I want to play a game in this world right now. I could turn off the UI here. Now I'm immersed in this world. This is very, very cool. I love what this looks like. Let's generate one more just for fun. Realistic. A modern day kitchen in a well lit house. And look at this. We got a modern kitchen in a well lit house. Kind of funky. It looks like the oven is over where, you know, your living room should be. Doesn't quite know what the kitchen layout should be. It's close. It's tried. There was an attempt. It looks like a modern day kitchen. You know what it was going for and it looks pretty interesting. It's just. I don't know if it's. I would lay it out that way myself. Here's one that I came across on Twitter from 360 degree bowling ball. Crystal ball. He created this 3D interior kitchen using blockade labs. And it kind of looks like it put this stuff where it was supposed to be a little bit better in this version. In a past video I talked about this company, Wonder Dynamics. They have a really crazy tool coming out where you can record real life Footage of a human, and then replace that human with a 3D computer generated graphic, right? Like this robot walking through was really this person walking through the tool, found the person, found the skeleton and replaced the person with the robot that we just saw there. This is called Wonder Dynamics. It's still wait list only. We don't have access to it. But there's been some new videos coming out showing off what it can do. Like this one, a robot with a boombox head. But it started as that person looking around, or these people that were jumping, or this person that was climbing a rock wall that they converted into a robot, this person dancing, and they turn the dancer into a robot or something. That's another video that just came out using Wonder Dynamics. But then yesterday I saw this. Anybody who's familiar with Corridor Crew will recognize this guy. It seems that Corridor Crew has gotten early access to play with this. And it makes me really jealous. I want access to play with this. But I like the fact that somebody other than the creators of this tool are now putting out videos showing off what it can do, because it means they are actually giving some people access to it.

Hello, human. I am here to visit all of you guys here today. It's all kinds of crazy stuff with us here. I mean, what am I gonna be doing next, right? I don't even know. We could do all kinds of dance party stuff. Kyah. I don't even know anymore.

Yeah, come at me.

So that's super cool. That was posted on Twitter from Ren or Surrender from Corridor Crew. You can see in the bottom video, that was his footage that he was filming. In the top video he was replaced with a computer graphic that was doing all of the same moves. Again, I just like the fact that somebody outside of the Wonder Dynamics company is showing off the tool because up until now we've only seen their little demo video and now we're actually seeing other creators like ourselves creating stuff like this. I absolutely love the world of the visual arts and the graphic arts and computer vision and what AI can do to replace people and how we can create AI generated worlds and use nerfs to create AI generated areas that we can walk around in and how this is going to overlap with gaming and videos and virtual tourism and all of this stuff that's in the works. This is all super fun and exciting to me and I'm excited to play with some of these tools when I can finally get my hands on them. And the future looks bright. This looks like some really, really fun, exciting stuff that I can't wait to get my hands on and play with and I wanted to show them off to you and nerd out about them for a minute because I know you'd probably be interested in this kind of stuff too. And if you are interested in all these kind of nerdy futuristic tech, AI and visual art tools and all that kind of stuff, check out Futurtools IO. This is where I share all the cool tools that I come across. Right now We've got over 1300 tools on here that you can sort and segment and find exactly what you're looking for. But if that's too many tools for you and you just want the TLDR of the Week, join the free newsletter and every single Friday I'll send you an email. In that email I'll share just the five coolest tools that I came across for the week. I'll share a handful of news articles, handful of YouTube videos, and one cool way to make money with AI. It goes out every Friday. It's the TLDR of the week in AI and just some of the coolest stuff that I came across and you can find it over at FutureTools IE and clicking the button to join the free newsletter. There might also be a button on YouTube below this video somewhere for it to but I highly recommend it. Also, if you like seeing stuff like this video inside of your YouTube feed, make sure you give this video a like and a subscribe and that'll make sure that you see more videos like it in your feed and you'll see more videos from me. I really appreciate it. Thanks so much for nerding out with me today. I do really appreciate you. I'll see you guys in the next video. Bye bye.

Sam.