Cool AI Tech Stuff I Think You Should See

Every once in a while I want to make a video where I just kind of share all of the cool stuff that I've come across. So this is Cool Stuff by Matt Wolf. And this video is sponsored by Shopify. So thank you Shopify for being a company that helps me create fun videos like this. I got a lot I want to share, so let's dive in. This is some cool research out of the Hong Kong University of Science and Technology called Animate A Story. This allows you to actually tell a story and then create an animation from that story. And the way it does it is interesting because it uses existing video and text to video and combines the two to create the story that you're trying to generate. You can see it says we leverage an off the shelf video retrieval system and extract video depths as motion structure. Here's an example that they share. I'm gonna go ahead and mute the audio cause I don't know the licensing of the audio. But you can see they've got a teddy bear here. A teddy bear wakes up in the morning in his bed. He rides his bike to attend a meal with friends. Then he walks into a big palace and starts to wait his friends in front of a window. He and his friends start to chat. After the meal, the teddy bear walks street to go home. He encounters a lost Santa Claus who is asking for directions. Then he walks through a park. The park has a beautiful waterfall. Finally he returns home and goes to sleep, ending his happy day. The end. You can see that throughout the video. It's actually a pretty consistent character. The bear stays the same color. There's a few scenes where he seems to lose his eyes. But overall the bear looks pretty dang consistent. Now if you use other text to video generation models, a lot of times it's really hard to maintain the consistent consistency and look of the character that you're trying to create. So this is pretty cool research that's coming out. And it appears if we go over to GitHub, the code isn't quite available yet. But I imagine since they set up this GitHub repository, we should have access fairly soon. Now here's some research called Animate Diff. Animate your personalized text to image diffusion models without specific tuning. Now I'm not 100% sure, but I think this might be the technology that's going on behind the scenes of Pika Labs. But if we take a look, basically they're giving an image and if I click on the image here, you can see it gives a quick little three second Animation of these various images. Here's another one where you see the wind blowing in the hair. Another one where it's this sort of cyberpunk character. Here's a little chipmunk or squirrel eating and dude moving his head. And this is essentially a model where you can give it an image and it will animate the image. And if we look at the GitHub page here, it is available to use. And also it looks like it can be combined with a prompt and a negative prompt, as well as a guidance scale and steps very similar to if you were to generate an image with this Pika Labs tool. And some of these examples are pretty cool here, like this little bear playing an accordion, or this chipmunk licking a hamburger, or this raccoon playing a guitar here. Now, again, this is available on GitHub. There are some instructions to install it. Or you can just use something like Pika Labs, where if you join the beta, they'll give you access and then you can simply jump into their discord. Use the create command here. Upload an image. I'll use this image of a mountain scene here. I could just drag and drop it in here and then add a prompt like moving clouds and rippling water. And we get something like this, which is very subtle, but if you look at the water, you can see the water's rippling, the clouds are moving very slightly, and it almost looks like the camera's panning a little bit. I don't know for sure if it's using the same technology, because with Pika Labs you can also do just straight up text to image without giving it an initial image prompt. Either way, this animate diff looks interesting and looks pretty promising based on some of these animations that I can see it doing right now. If this isn't what Pika Labs is using, it's only a matter of time before there's tools out there that use this behind the scenes. Speaking of Pika Labs, since this is a video about me showing off some of the cool stuff that I've come across recently. I've come across some really cool videos that people have made with Pika Labs, including this one from Alan T. Here of these giraffes playing guitar together. Here's one from Von Klepp where you can see an animation of these guys in gas masks and it says destro us. I'm not playing the music because I don't know the licensing to it, but some pretty cool animations here. Here's one of my favorites from Scotty Wick where he actually made Futurama as a cyberpunk movie here. And you can see here's Fry coming in through his little frozen thing from the start of Futurama, exploring the new futuristic city that he just found himself in. There's the professor there. That must be Bender. And then my favorite part is right there we have Zoidberg that I think is supposed to be Leela. That's Zap Branigan. So pretty cool animation. I love this idea of trying to turn cartoons into some sort of realistic version of the cartoon. Check out Scotty Wick. This is a really cool video that he made. Here's some research that came out of bytedance Inc. Called Bubble GPT. This is another one where the code is actually available on their GitHub page right now. But this is a multimodal large language model. So here's an example where there's a picture of a cat. Please describe this image. The image shows a cat wearing sunglasses and sitting on an orange float in a swimming pool. Is this image funny? And why? Then it explains why it's funny. And then you can see how it actually broke out the various elements of the image right within the chat. Here's another example. Hi Bubbo GPT. Can you describe this image? The image is of a red deer stag standing in a field during sunset with antlers and thick coat of fur. The stag is looking towards the camera with a calm expression and etc. Etc. You can see that it's actually breaking out elements of the image and using the computer vision to sort of detect various things inside the image. To answer the questions, here's an example they give of giving audio to the chatbot. Can you describe this audio?

In this audio we can hear the distinct sound of dishes being manipulated in a kitchen setting. It could be someone opening a cupboard, stirring a pot, cracking an egg on a pan, and then plating it. The ambience portrays a domestic scene with the person involved in cooking or preparing food. And here's one with aligning audio and image. Try to find the sounding source in the image according to the given audio and generate a detailed description about.

It. Then responds. Based on the provided image of a boy riding a bicycle, it appears that the source of the audio is the sound of a dog barking. The boy appears to be on a dirt road, likely traveling through a wooded area as the tree branches are visible, etc. So it actually managed to look at the image, pull out the various items within the image and listen to the audio and then give a Description of what it hears and sees and combines the two. So it looks like right now it can handle text, image and audio, but not quite video yet. However, there is this cool research demo that's available on Hugging Face Spaces right now called Video Llama, where you can actually upload a video and then ask questions about the video. Now, they do have some examples here, like this video of a dog on a skateboard here blaming himself for meanwhile our skateboardy dog out on the portwalk. And then we can come up here and enter a prompt of what is this dog doing? The dog is skateboarding on a sidewalk in the street. What colors is the dog? The dog is black and white. What type of dog is it? It's a small dog. What breed? It is a collie. So it's actually getting all of this from looking at the video here. Let's see what happens if we upload our own video here of a monkey eating a pineapple. This is an AI generated video, so I have no idea how this is going to work, but let's go ahead and upload it and start a chat. What is going on in this video? In this video, we could see a monkey sitting on a sandy beach eating a piece of fruit. The monkey is wearing a hat and is using a pair of pliers to hold the fruit in its mouth. The beach has a blue sky and some palm trees in the background. Now, I don't know where it got the pliers, but you can see at one point there is a pineapple on the side of the image. And I think that's what it sees as the palm tree. No idea where the pliers came from, but pretty cool that we're starting to get tools where we can actually have conversations with videos and ask questions about what it's seeing in the video. Now combine the last research we just saw with this research where it can listen to the audio, see images, see videos. You start to pull all these together and now we've got true multimodal, where you could just upload any type of file and it will help us describe and understand what's in that file. Now, here's some research out of meta called CodeTracker. And basically what this does is it finds a whole bunch of points on the video that you're trying to track and then tracks all of those points at once instead of trying to just pick one single point and track that, because that's a lot less accurate. This is also one where Meta did make the code available over on GitHub so you can actually run this locally if you want or inside of a Google collab here. Personally, I just find watching some of these clips really satisfying to see the sort of rainbow colors track along with what's going on in the video. So here we have some previous examples of similar trackers and you can see this sort of pips tracker, it doesn't quite track properly. A lot of extra stuff seems to get tracked that shouldn't be tracking. Same with Raft, same with this taper. But then when you see code tracker you can see in these examples that it's only tracking the stuff in purple and a lot of the extra stuff that it shouldn't be tracking isn't getting tracked with it. They also have some examples of individual points being tracked versus co tracker. The results here are a little bit less dramatic than what we see on the points on a uniform grade example. But all said and done, this is just a way to make tracking specific subjects in a video a lot more accurate and efficient, which is going to make it a lot easier for video editors to mask things out, change variables on the subject of the video like shirt colors and and things like that, because it will better follow along with the subject in that video. Now I want to quickly talk to you about Shopify. Shopify is today's sponsor of this video and if you're not familiar with them, they're an online commerce platform that makes it easy for anybody to sell their products online. They're the platform that I'm using to build out the future store where I'll be selling cool mugs and shirts and hats and other swag related to AI. If you didn't know, Shopify leverages AI in several ways. In a recent announcement from Toby, the CEO of Shopify, he talked about the upcoming Sidekick that Shopify is about to roll out where you can use AI to assist you in building your store. For example, he asked the Sidekick, I've had a drop off in sales from March to now why? The Sidekick goes on to say the dip in sales is probably due to minimal snow. These are summer months in many regions, so it's natural for sales to decrease. He then tells the Sidekick, put everything on sale and then it puts everything in the store to a 10% discount. In this example, he also uses the sales assistant to add surfboards to the lineup and then eventually even converts the entire shop into a surf shop instead of a snowboard shop, all using this new Sidekick AI that Shopify will be rolling out soon. But not only that, but Shopify essentially has their own AI Search engine powered by the Shop app where you can search their various Shopify stores. So for example, if you go to Shop App AI, there's a little chatbot over here on the right side. What are you shopping for today? I would like to find some cool AI tech. Great. Here are some ideas for AI tech products. Smart speakers, robot vacuum, smart thermostats, smart security cameras, smart lighting systems. And then it shows me various stores that are built on Shopify where I can purchase those items. Let's tell it, I'm shopping for interesting virtual reality tech. What should I be looking at? It gives me some examples here. Oculus Quest 2, HTC Vive, PlayStation VR, et cetera. And then shows me exactly what Shopify stores I can buy them on. What is the least expensive VR goggles? It answers over on the sidebar and then links me to where I can find them. Cardboard VR $4.99 so while Shopify is a great commerce platform to build stores like the Future Store, they're also on board with the AI revolution. Building out some cool next level tech to make your online shopping easier with AI, as well as building your own online stores easier with AI. If you want to build a store over on Shopify and leverage some of the new tools that they're going to be rolling out soon, head on over to shopify.com Matt Wolf and for just one, you can get a trial for three months of Shopify, which should give you plenty of time to get your store online up and running. So thanks again to Shopify for sponsoring this video. So here's another cool research paper that came out recently called sketch a shape zero shot sketch to 3D shape generation. Now, I haven't found a GitHub page for this or any place where they're demonstrating it on video, but this looks pretty interesting to me. It's from Autodesk Research, which is the company behind CAD, one of the original 3D design tools. With this, it looks like we'll be able to create really basic archaic images. Like I would have even known that this was supposed to be a chair, but it generated this into a 3D chair. They created this image of a skateboard. It converted it into a 3D image of a skateboard, a wine bottle, and actually pretty good drawing of a guitar into, I guess, a less good drawing of a 3D model of a guitar. Now, it does leave a little bit to be desired in the 3D images that it's generating. It looks like it's doing a really, really good job of creating these 3D parts out of these sort of 3D drawn images. But, but when you give it fairly bad drawings, it does an okay job for what you're going to get out of it. Now, again, I don't know when we're going to have access to this. I couldn't find the GitHub page for it. But this will be something fun to play around with where we'll be able to draw These images, get 3D assets out of them, and then likely pull them into something like Blender and clean them up a little bit. Now here's some research that came out of Google called Hyper Dream Booth. And if you're not familiar with Dream Booth, I did make a video of it months and months ago. It's basically the tool that you use to train your own face into stable diffusion. You upload 20 to 30 images of yourself, it learns what you look like, and then you can use text to image prompts to generate more images that look like you. Well, this Hyper Dream Booth looks like it's going to be able to generate images that look like you based off of just one input image. You can see here's some other examples here where this was the input image. This is the image over here that it output. Here's another input image, here's the output of it. Here's an input image, here's some various versions that it outputs. And you can see some other examples down here. To me, they don't look quite as close to what you would get from something like Dream Booth. With that additional data of Training yourself on 20 or 30 images, I think you're going to get a much more consistent look. But the cool thing is we're getting closer and closer to being able to create these AI images of your own likeness with less work going into it. Now, speaking of AI images, this is a really cool product. I don't believe it's available yet. It's called Stacks and it looks to be something that leverages the Mid Journey API, which I didn't even know Mid Journey was allowing people to use their API yet. But if we watch their demo here, it looks like it's trying to provide a much better user interface for Mid Journey. You can see in his example here, he has the option to write a text prompt or speak out a prompt. He enters the prompt a barren sand and desert surface with a woman standing overlooking the area with broken down buildings. He can apply different text weights to it and it generates the image just like we would expect from midjourney. But then it gives a menu Here with blend color style title, Niji style quality Random seed stylize. Weird chaos. Zoom level. He clicks on zoom level, cranks it up to two, and you can see it zooms out on that image. But also over on the left here, it created a second layer to this stack. He then clicks the add button this time, clicks the random seed button. It generates a new random seed, generates a new image, and then it adds that new piece to the stack over here on the left. Now, he could rearrange the stacks, and it creates a sequence of images so he can sort of see the results of what he's been creating here. He then creates another prompt. Dead trees on the surface of the ground. It creates another image and adds it to his stack over on the right. And you can see here's a quick visual they have up on the website of the various layers that are available that can all be dragged and dropped and rearranged to change the order and the sequence of the images. So it looks like a really cool, really intuitive, almost better user interface for midjourney. I mean, let's be honest, almost anything is going to be a better user interface than what we're getting out of discord right now for midjourney. Now, I have no idea when this is going to be available or even if this is going to be available. This might even just be a concept right now that whoever created this website would love to see. But this is an app that I want. I hope this actually comes out. I hope this is real. The website is actually called fake up, so hopefully it's legit, because I would love to see this app in real life. Now, several weeks back, we talked about drag your gan, where you can select points on an image and then drag them around, and it will actually move that image around and reshape it to what you're looking for. Well, this is available now on Hugging Face and you can actually play around with the demo. I actually duplicated the space because when I tried to run it in the free hugging face space, there was too many people using it and the queue time was just way too long. So I duplicated it. It's costing me about three bucks an hour to use this right now, but you can still use it for free if you're willing to wait the queue times. Now, one sort of downside with drag your gan is that you have to use one of their existing images to see how it works. It has to be a gan generated image. You can't just upload your own image and drag it around but in order to use it, we can create a flexible area here. Let's say I only want her arm to move. I can sort of create a mask around just her arm right here and maybe kind of the space around her arm. And now you can see I've masked just her arm. Now if I go ahead and add points, I can click on her elbow here and move her elbow up here. Click start and it will slowly animate her elbow up for me. And it looks like I went too far because it sort of made her arm disappear. Let's go ahead and stop that. I'll reset the image here. Let's go ahead and pick a different image. We've got this car config image here. Let's click on the top of the windshield and let's move it up here. And click start. And we'll see it slowly animate, turning this into a much, much larger car. Now I'm actually speeding up the video because it actually moves very, very slowly in actuality. And now we have what looks like a totally different car. Let's see what happens if I grab a point on the wheel right here and move the point on the wheel forward. We'll click start. And again, this process is really, really slow. So I am speeding up the video a little bit. And you can see we now turn what looked like a small old car into something that looks more like a SUV truck hybrid thing. So that's drag your gan. It is available on Hugging Face Spaces as of the recording of this video. If you just click on spaces at the top, it's one of the spaces of the week right now. Drag gan. Drag your gan. But if you don't see it in the spaces of the week, just search drag gan and it should show up. Here's another cool tool that I came across this week called Swap anything. You get a certain amount of credits to use it for free. But you can upload any image you want here, mask it out, and change what's going on in the image. So, for example, I'll upload this image of myself here. I could select where I want it to mask out. So let's say I wanted to mask the background. I'll confirm this is the mask. And let's say in front of a fiery volcano and click create. It'll take a moment to process. And now here's me in front of a fiery volcano. And it gave me two variations that I can play with. Let's go ahead and click on reselect to change the mask again. I will remove the mask on the background. This time let's add the mask to my shirt and then let's just type wearing a tie dye shirt. And you can see it converted my shirt into to a tie dye shirt. Now this is another one that's free to use. Anybody can use it. Right now it's at this weird URL, but I'll make sure it's linked below in the description. Here's something I recently came across from Alpha Signal AI here. This is a customer service chatbot called Air, and it's a chatbot that's designed to sound like a natural realistic voice to handle customer service for your product. Now there is a slight delay. I'm not gonna cut out the gaps so you can hear the delay. Cause the delay really gives away the fact that it's not a real human. And this is the worst it's gonna get.

Hello?

Hey, James. Yeah, who's this?

Hey James, it's Alexander from Tesla Motors. How's your day going so far? It's going pretty good, my man. How about yourself? You said from Tesla, right? The car company?

Yeah, that's right. I'm calling from Tesla Motors, the car company. Did you have any trouble or questions while customizing your car on our website.

So you can watch the full video? I'll make sure this tweet is linked up below as well. But the conversation kind of, it continues like that with the gaps between the real person responding and then the response from the actual computer. Here's some more cool tech that I came across from Brian Romley's Twitter. This is an AI based Apple picking drone. The video quality isn't great here, but you can see that they've got these drones that will fly up, find the apples, pick them, and then pull them down and drop them in a truck. You can see here's what the computers see. They actually manage to notice what's the apples and what's not the apples and only pick the things that are the apples. They fly, they grab them, they drop them in. Speaking of cool tech, check out this mixed reality car tour that was shared by Timmy Guerrero over on Twitter. So you can see here's a car, but the car's not really there. He's got this mixed reality headset on. You can see it's an empty room and the car starts to form in the room. And then he can actually take a tour of the car. He's looking around on the inside of the car, seeing all the of the equipment, looking around on the outside using the mixed reality headset. He can do a full virtual tour of this car without the car ever even being in the same room with him. Speaking of cool car technology, check out this car with color changing paint that I came across from my buddy Barsy here. This car actually changes its paint color and it actually uses E Ink, which is actually what is used in something like a remarkable tablet or a Kindle where there's a thin layer wrap around the entire car that uses this E Ink. And the E Ink can then change colors based on the user input. So this is pretty cool technology. I know this has nothing to do with AI, but you know what? I'm a tech nerd and this is some cool tech nerd stuff. Here's another example of the same technology where you can see the color on the car actually changing and almost like rippling like waves across the car. Now this technology comes from BMW and I believe this is from the Consumer electronics show from 2023. This is going to be really interesting when we see this kind of stuff. Driving on the road also seems like a real distraction to other drivers if your car is lighting up and changing colors as you drive around. And that's all the cool tech that I've come across from the last couple of weeks that I wanted to show off and share with you. I love nerding out about all of this stuff. It's not only AI. I love talking about AI, I love talking about anything future tech. There's a reason I didn't call my YouTube channel mat talks AI. There's a reason I didn't call my website AI tools. I just love future tech. And this video broke down some of the coolest future tech that I've come across. Now before I wrap up, I want to say thank you. Recently posted this video on Twitter where I shared that I recently received an award from Tubebuddy called the Emerging Creator Award. Here's the actual award that they sent me. You can actually see it over on my desk over here. It's basically an award for being the best emerging commentary creator on YouTube. Whoever nominated me, whoever voted for me on that. I don't know how the nomination or voting worked, but thank you so much. I'm so appreciative. I really honestly never thought I would be in a position to have a career as a YouTuber much alone actually win awards for doing it. Thank you so much for everybody who watches this channel, for everybody that nominated me or voted for me or however that worked, I am so, so grateful. Thank you so much. I really appreciate you checking out this channel and nerding out with me a few times a week on all the latest cool in tech and AI. And finally as I always do in every video I gotta remind you, check out FutureTools IO. This is the site where I curate all the coolest AI tools that I come across. It's completely free. I'll keep you in the loop with all the latest AI news, all the latest A tools and if you join the free newsletter, I'll send you one email every Friday with a breakdown of all the cool stuff I came across for the week. All that's available over at FutureTools IO. And once again thank you so much to Shopify for sponsoring this video. Companies like Shopify are truly helping me live my dream of being able to create fun stuff like this for a living. So thank you so much and thank you to you for watching this video. I do really really appreciate you so thanks so much. I'll see you in the next video. Bye bye.