EP 218: Winning the Probability Game in AI Visuals

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

If you want better AI images and better AI videos, sometimes you gotta roll the dice and play the probability game. But don't worry, you don't have to do it blindly. We have an expert today that's gonna share his secrets and hopefully allow us all to get better AI images and video. So what's going on, y'? All? My name is Jordan Wilson and I'm the host of Everyday AI. So we're a daily live stream, podcast and free daily newsletter helping everyday people learn and leverage generative AI. So if you are joining us on the podcast, we appreciate that. Make sure to check your show notes. Today is going to be one of those shows that's a little more visual, so you might want to check out the show notes and watch the video. I think we're all going to learn a lot. If you're joining us on the live stream, thank you. We appreciate that. I'm David joining us and Tara joining us from Nashville. Thank you. So before we get in and we're going to walk everyone through it, I think today is going to be a very fun, very visual and interactive episode. But before we get there, let's start as we do with the AI news. And as a reminder, you can always go to your everydayai.com and sign up for that free daily newsletter. Unlike the other guys, we're actually written by humans. I'm a human. I write the newsletter. So I'm a human now and I'm gonna read the news. All right, so here's what you need to know for AI news. So Meta's Llama 3 is reportedly in the works. So Meta is set to release an open Source language model, llama3 3. And llama3 is designed to be more responsive to users and provide context for difficult topics rather than blocking queries. It'll also have the ability to differentiate between words and sensitive or harmless meanings, depending on context. And then llama3 is estimated to have more than double the parameters of its predecessor, llama2. All right, our next piece of AI news. Speaking of AI images. So Google CEO Sundar Pichai has publicly apologized to employees for the Gemini AI image debacle. So Google CEO Sundar Pichai has issued the apology for the release of the company's artificial intelligence tool Gemini, after facing backlash for its bias and sometimes offensive results. So the tool was meant to create diverse images with the built in Imagine 2 image model, but instead produce images of, as an example, America's founding fathers as black, the Pope as a woman, and other historically inaccurate and sometimes biased images. Google has also announced that it's planning to relaunch the AI generator in a few weeks. All right, our last piece of AI news, at least for the podcast here. Did the New York Times hack OpenAI? Well, that's what OpenAI is alleging. So in a new filing, OpenAI has accused the New York Times of quote unquote, hacking their products by using deceptive prompts to generate copies of the New York Times articles in violation of their terms of service. The New York Times had filed a lawsuit against OpenAI for copyright infringement, showcasing how OpenAI's GPT models can produce verbatim copies of their content. But OpenAI did claim that hacking was a rare error and in their systems learning process, which could be addressed. The New York Times obviously has been arguing that their actions were merely a search for evidence of copyrighted content within the AI models. And there is an ongoing debate between OpenAI and the new York Times regarding the use of manipulated prompts and using copyrighted copy. So super interesting if you're interested in that. I actually had a full like one hour video, very deep dive. I don't know anyone else that went in as deep as I did, so I'll make sure to leave that in the show notes so you can go take, take a look. But there's always more. So make sure to go to your everyday AI.com and sign up for that free daily newsletter. All right, I'm excited. We're not going to talk about AI news all day. We're going to talk now about how you can improve your AI visuals. Right? So if you've ever used an AI image tool like Mid Journey or an AI, an AI video tool like Runway or Pika Labs, you know, maybe you got great results, maybe you didn't, but our guest today is going to help us hopefully get much better results. All right, so let's go ahead and bring to the show. There we go. We got him. Got him live here. So tnu, a visual artist and founder of Tyai Tianyu. Thank you for joining the show.

Hi Jill and hi everyone. Thank you very much. This is Ken from. Based in Singapore.

Yeah, yeah, well, it's, yes, very good morning for me. It's, it's a good evening in, in Singapore there. So thanks for joining us halfway across the globe, tnu, but maybe tell us a little bit about what you do as a visual artist. And founder of tyai.

Yeah, actually my background is quite far away from artist. I'm. I don't. I won't call myself an artist even now. So I have a background in market research, social media analytics and advertising sales. So I ventured into generative AI almost at this time last year. It's very addictive to me personally, and especially when you create the visuals and videos and chatting to ChatGPT. Last year I consulted in market research, data analytics and content production. Of course, with the help of AI and only until the end of last year I discovered that, well, education should be my focus because eventually generative AI will reach everyone and it's an essential skill for anyone to be successful in the future workplace.

That's so important because I think Tianu, there's, there's a misconception, right? Because a lot of people think, oh, generative AI is not for me. But I don't think, you know, my personal take is you don't really have an option. Right. So whether it's in, you know, two months or two years, I think the average person, even if you're not a creator or marketer or, you know, a visual artist, you're going to be using, you know, some sort of generative AI tool. Is that kind of like your, your thought on it? Is that kind of where you see things going?

Yeah, that's, that's my view as well. And I know it. It can sound quite, it can sound very, you know, it can sound very complex if, especially when you hear the term prompt, engineering. There's a prompt and there's engineering. It doesn't sound like something that everyone will do, but actually it's just the, it's just communication. It's just a way to communicate with AI models. You used to communicate with people, with humans, and now we are communicating with AI models that in that way, I think eventually everyone, everyone actually has a potential to be a good prompt engineer and everyone should be ready to learn them, to learn these skills.

Absolutely. So I'm curious, how did you transition from market research to now? You're putting out very fascinating and great visual content and we'll be sharing that in the newsletter as well. So you can go look at a lot of his work. But how did you make that transition from market research to now all of a sudden you're putting out amazing and educational content on AI visuals.

Yeah, this is a great question. Even if I'm doing the AI visuals, my methodology is still like market researcher. I study the statistics, I study how the models are built and Then I look for the areas where the models are particularly good at, and then I take an analytical approach when I design the prompt and eventually optimize the prompt. So whatever I'm doing, like, in the past 12 months have actually have a strong connection to my past experiences.

And, you know, I'm curious, how did you get to the point where you are breaking it down at such an analytical level? Because I feel a lot of people with AI visuals, you know, they'll just go in there, you know, play around, try it once or twice, and, you know, then they kind of give up and they're like, all right, well, you know, it's okay, but maybe I'll just wait for the technology to get better. So how did you really push yourself to, you know, go in and really just break this down, you know, like a. Like a market researcher? Like, how did it get to that point?

It's very. It's very exciting as a, as a market researcher. It's like a complete new field for you to venture into. And you. And there's no manual for that. There's no. Even if you look at the Official documentation from OpenAI, from MidJourney, from the other models, there's no manual for you that teaches you to talk to the model step by step. The only way to learn is to experiment. I start with a very fundamental. I start with a very basic prompt and eventually build up all the variables and all the complex prompts and structures. So I. Yeah, I just start from the very basics. I think that's the. I think that's the way to talk to almost every large language model or.

Image model that's hitting me right there, right? Like, I had a whole episode yesterday, you know, kind of recapping everything about ChatGPT, and I think that's a perfect explanation, is you should always start very simple with natural language and kind of see your results and go from there. So. So I'm curious, tnu, like, how many, right? Like, when we talk about generations and we'll talk through that process of what that means, but, you know, on average, you know, did you start by doing, you know, two generations, and now are you up to like, 10, 20, 50? I mean, in general, when you're trying to get that perfect, you know, maybe AI video out of, you know, pika labs or something like that, how many generations are you sometimes doing?

It depends on model, depends on the subject. So if I post a lot of things about cats, because it's really easy to do, and I just need to generate a few images of the cat of a certain style, then it is good enough for me to showcase different art styles or different. Different things you can do with Dall E or with other models. But they are more complex topics. So for example, if. If the model is not well trained on a subject. For example, midjourney before v6, mid journey was unable to create mermaids or scorpion. So maybe you need to try 100 times to get a scorpion. Yeah. In that case, you will need more iterations for the video model. It also depends on the subject. Some of the subjects are quite easy. For example, Runway and Picard are both very good at the natural things. For example, the water, moving water, waterfall, anything with water. You just need to create one or two videos and then they will be almost perfect. But if you want to create something different, if you have an image, for example, I have a cat. I always use a cat. So I have a cat kayaking on the white water. That could be difficult. That could take more tries. Maybe it'll take 20, 30 tries to get one video right.

It's funny because if you're listening on the podcast, I'm kind of laughing. So same thing for me because, like, even the original, like, AI image generators, like, I did like cats too. Like, I was trying to, you know, hey, shout out my cat Rocky. Right. I was trying to generate pictures of him. Yeah. Something about cats is just like, you know, fun. So a question here from. From Monica. So thanks for this question. So asking, can you explain a little bit more in depth on what generations are. Yeah, we should probably explain some of this terminology, but yeah, what's a generation? And you know, when you're talking about doing them over and over, does that just mean that you're running the same prompt over and over or are you tweaking things each time? So talk a little bit more about what it means to go through another generation.

Yeah, sure. This is a great question. So in generative AI, generation simply means generation of an image or generation of a video. We don't tend to use create a video because you don't actually create a video. You generate the video, generate the image with the model. Yeah. And then. Sorry, what's your question again?

Oh, no, that was perfect. Just explaining those, because I think a lot of people maybe on our show are using large language models a lot more. Some people might be using Mid Journey or something like that. So I think it's important that we just kind of talk about what these models even are and what they do. Right. Because. And maybe it's just good to explain to the audience. So you have your, know your AI image generators where you can put in either a, a text or a photo and get a photo on the back end. So you know, photos like Mid Journey and Dolly and you know, kind of what we're talking about here a little bit more is you know, these AI video, you know, companies like, like Runway and in Pika Labs where you can either put in text and get video or put in photos and get video. So I'm, I'm, I'm curious Tianyu, like what's been your kind of best process? Like do you use photos, do you to get videos? Do you use text to get videos? Do you sometimes test to do both and see which one's better? What's kind of been your, your best approach for generating videos in terms of what you start with?

I, I first tried the. So there are only two ways to generate a video. One is text video, which is very similar to how you interact with ChatGPT or MidJourney. So basically you put a description, the prompt and then the video like typically is about 3 seconds or 4 second video will come out based on the prompt. That works on stable, stable diffusion video P or Runway. So these are the leading models before Sora. So when SORA comes out, everything's changed. Everything changed. Right. So we, we are still talk. So I'm, I'm still talking about some, some common practices I do when in the models like Runway or Picard. So I discovered that text to video is not that robust. It's okay if you just create a short video or a few short videos and you can combine them and make a short, a very extremely short film out of the text to video models. But the consistency is always a challenge. When I say consistency, I mean the consistency of the characters. For example, if you have a video of, you can use a text prompt to create a cat kayaking on white water. And then you need to create another, it's only four seconds. So each generation you only have a four second clip. Then you then to make a longer video, you need to generate another one. And then your prompt, you put in the prompt a wife cat kayaking on the, on the, on white water in front of the waterfall. And then you might get a completely different white cat from, from the, from the first video. So in that way it's almost impossible to make anything consistent. That's why I, I, I, I, I do not use the text to video methodology. What I, what I, what I always do is to create all the, it's like building a Storyboard. So I create all the images with dall e or with other models with somewhat consistent character first and then I upload the images to Pika or to Runway and animate every single image, making them a 4 second or 8 second video and then combine them so that they look more like a consistent mini film.

Yeah, no, it's, yeah, that's, that's a good point. It's something to, you know, point out there tnu like. Yeah, like, you know, we talked about OpenAI sora that you can, you know, put in a text prompt and get up to a minute of video. Right. And then you don't even have to do these multiple generations because it generates multiple scenes together. But you know, the majority of people out there do not have access to that yet. To that yet. So, you know, we are all having to go through a similar process that you're laying out here. So maybe what we can do is kind of walk our, our live stream audience through this and maybe show them a little bit of what you're talking about. And again, if you're listening on the podcast, we'll do our best to, to talk you through this a little bit, but you might want to check out the show notes and come, come watch this. So let's go ahead and tien you, maybe you can walk us through here. You know, so we have Pika labs open, right. And we're going to be generating a four second video. So let's. Yeah, maybe just go ahead and walk us through what you're doing here and then we can talk a little bit about your methodology.

Yeah, sure. So right now I'm in Pika. Pika is, yeah, so Pika within Pika. You can do the text video or image to video. And here I have uploaded an image of a white cat kayaking the white water. So here we are going to make, we are going to animate this image, making it into a three second video. But we have no clue that which direction this video can go at the beginning of the process is very similar to rolling the dice. Basically, it means that you need to try every direction, you need to try every parameter to see which one actually work. Well, Pika has a very good camera control feature. Means, so you can, means that you can, there's a virtual camera that you can control the direction of the camera so that the image will move as a video. The image will move according to the camera movement. So for example, right now I'm at pan left and pan right. Each time when I roll the dice, I try to Just roll. Just choose one direction. Just use one. One variable because it's easier to optimize in the future. So let's try pan. Right. And then you can also control the strength of motion. Typically, the lowest motion is 0 and the highest is 4. But normally, based on my experience, the higher strength of motion never work. So I only choose. Let's choose one strength of motion as one, and then that's it. Then we can generate the video. So that is only one video. We never. You never know, right? You never know whether it's going to work. So, and. And then. Then we should try the next. The next parameter. How about pan left? So we can do the same. Everything remain the same, but the only variable is pan left. And then we create another video. The same thing works for tilt up and tilt up.

I'll give a little bit of background here. If you're used to maybe using something like ChatGPT, sometimes you have to wait. But with Pika Labs and a lot of the other AI video generators, what Tianyu is doing is he's able to, you know, generate maybe 4, 5, 10 at a time. And you kind of have to wait for them in the background. Right. So these different generations, kind of his. His rolling of the dice, they're all kind of slowly loading one by one, and then we'll be able to see, you know, which of these directions that. That he kind of put in there, which is going to work best for this specific photo. So one. One thing that, you know, I'm curious about is, you know, you said that you always start with, you know, like a one motion, because if you go much higher, you know, it might not work very well. And you also kind of first test, you know, different directions, I guess, is, is that the best way for people to go or maybe should they, you know, as an example. Oh, let's test, you know, camera right at one, camera right at two, and camera right at three. Right. Like, I guess. How did you come in and get to that? This is the best way to start the process.

Okay. We can. Since we are doing the live demo, we can. I can show you what. How would the different strength motion look like? So we do pen, right, with a motion. Let's do the extreme one. Let's do pan write with a strength of motion of 4, which is the fastest? Let's do the fast and furious. Yeah. And see how it works.

All right, great. The original ones that you did with the different areas of motion, all with the one. It looks like they're done. Let's Walk us through and show us how you can decide what's good and what's not.

We already have pan right. The first one is pan right. It seems okay, right? It seems okay because the kayak is moving from the left to the right. And if your camera moves towards the right, it seems like a natural movement. And. Yeah, and I think this one is all right. How about pan left? Okay, so pan left. Do you see the problem? If you do the pan left, the water also moves to the left that goes the opposite direction.

Yeah, that one looks very unnatural. Looks a little wild, right?

Yeah. This one doesn't work. And how about this one? This one is tilt. I think this is great. This is one. This one is tilt up and it's showing the kayak going up. Going up and down. Yeah. I think this direction also works quite well. And the next one is down.

Yeah, that one looks a little weird. It looks like something ominous is about to happen to the poor cat. Kayaking.

Yeah, the cat is talking. Maybe we can do the lip sync to the cat.

Oh, yeah, I see that. Interesting. Yeah, that's so, so that's important to talk about. Right. So with. With some of these, you know, you know, video generators, you don't have always a ton of control over the actual movements. Right. So in that instance, the cat was actually just moving its mouth where in the other instances it wasn't. Right. So, yeah, you know, I guess. Have you found a way, you know, aside from. And we can talk about, you know, maybe Runway and their, you know, multi motion brush. Right. But is there any other way to. To get that consistency or is it just. Man, you just got to keep. Keep doing generations until you get it exactly right.

Yeah. Sorry. Before I answer your question, I want to show you this. This is a speed of four. The, The. The strength of motion is four. And you can see that it's totally off.

Yeah, yeah.

There's.

Yeah. A wave crashing. The. The white cat on the kayak, wave crashing him. Mouth is going. Mouth is going crazy. Right?

Yeah.

Probably something you would never use there.

Yeah. And the cat can turn into a completely complete different animal.

Yeah. Okay, so. So essentially the higher we. We crank the motion there, in general, at least how the technology is now, probably the less usable something becomes.

Yeah. So based on our seven or eight experiments here, we can see that this one tilt up seems to be the best direct the best movement, the best direction for this image. So then what I will do next is to, if I'm a perfectionist, I can continue generate the same. I can continue generating the videos with the same setting multiple times, maybe 10 times just to get a perfect video. Or I can add four seconds, for example, I can extend the video to make it eight seconds. Or I can make other, I can edit, I can change the region, change different parts of the video to make it more interesting. Yeah. So in that sense it so rolling the dice is not just to roll the dice once, but after rolling the dice you find the right direction, then you double down on that direction, then optimize your prompt and then you continue roll the dice. So it's a chain of the activities.

And I love this. And Juan, thanks for this comment saying the same thing. Love the hands on and visual with these real life examples. A couple, couple questions here. You know, from, from Tara, a great question here and let's just go ahead and we'll go back to here. There we go. So, Tara asking, could you please share your strategies for organizing your media outputs? Specifically, I'm interested in how you utilize tagging or memory aids to streamline your process for future projects. That's a great question because, yeah, if you're doing, you know, you know, dozens of generations, maybe for the same clip, how do you keep that organized?

That's a great question. I'm terrible at doing this. I organize them based on projects. I have different folders for different visual projects. I also collect all the important prompts, all the custom instructions for ChatGPT. I collect all of them so that I can reuse them in the future.

Another, another good question here from Cecilia. So, and you know, I'm not even sure of this one, so maybe, maybe I'll be learning something. So, Cecilia asking, can you ask, you know, these, these different AI videos for a storyline after you create the video or do you retain your creative rights? Yeah, so like after you get that short one, do you, you know, can you go in there and then use a text prompt or do you just kind of keep doing what you did and, you know, Keep adding on? 4 seconds.

Are you still running in circles trying to figure out how to actually grow.

Your business with AI?

Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on Genai. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use Gen AI. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you can partner with us too. Just like some of the biggest companies.

In the world do.

Go to your everydayai.com partner to get in contact with our team or you can just click on the partner section of our website will help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on Gen AI.

So I think there are two parts of the question. The first one is do you how if, if you want to con, if you want to continue the story and you can keep, you can Continue Add on 4 seconds for, but you can only add on a few but you can only add on a few times. A few instances maybe add up to 16 seconds. So eventually you still need to create multiple clips from different images or different text prompts and to make them into a full film. Then in terms of the creative rights, it depends on the platforms. So for example, for Mejourney, there are certain tiers that your own full rights of the images. For Dall? E, I think Dall? E has a for, for Dall? E as well. So the users who create Dall? E images own the rights of the images. And then for the other image creation tools or video tools, it all depends on their terms of use.

Yeah, and that's important to read those terms because they're always changing and sometimes, and I've talked about this on the show before, sometimes they're a bit confusing or perplexing. Right. You know, passing off, you know, all, sometimes passing off all liability to the end user. So that's great, great, you know, great piece of advice there, you know, to always read those terms. All right, so another great question here from Yogesh, former guest on the show. How, how you doing, Yogesh? So asking, have you found a way to create images with text embedded? Yes. What's, what's the best way to do this tnu or is there not a great way?

Yeah, this is a great question for the rolling the dice approach because none of the image models are very good at embedding the text into the image. None of them because they are not trained to do this. But if you are lucky on Dall Ejenny, even on Google's Gemini, you can generate images with text with a simple text, simple, like less than five words. Then if you generate less than five words, you can get everything right within like three to four tries. So if you Try multiple times. Like if you try 10 times, for sure you will get one satisfactory image with the text. Less than five words with the text properly displayed.

Yeah, yeah, I, it's, it's almost like a, a painful process. Right? Especially with the text because you know, you get it so close and then you're like, oh, just got to keep, you know, keep generating and regenerating until you get it. I'm curious, I haven't tried this, like have you tried doing a, you know, something like Yogesh was asking there, you know, something with text but then creating a video afterwards. So if you get a good, you know, image that has text on it and then trying to create a video. I haven't tried that. I'm curious if you've tried it and if it, if it works at all.

Yeah, it works, it works. It depends on it also. So it's easier to create text on the images than to animate the, the, the animate the image with a video tool. Because the, some of the video models tend to have very limited, very, very limited, very limited fonts. So for example, I have something like a welcome written on the cloud. This image is generated with Dall E and then that word looks like the shape of the cloud. And then I upload the image to Runway. Then easily these words, the word welcome can turn into the same word welcome but in the font of Ariel, in the normal font that you can find in the word document. So that's the limitation there.

Yeah. And so I'm curious and we'll make sure to share maybe you know your, your favorite project in the newsletter. So everyone listening, make sure to check the newsletter out. Sign up at your everyday AI dot com. But I'm curious, tan you like, what is the most time or the most like generations that you've spent on a single project? And then maybe, you know, talk a little bit about that project.

I, I spent a lot of time at the beginning when I first used Generative AI. When I first used Midjourney, I spent a lot of time trying different directions and I was on my phone all the time on Discord. But then after a few months I figured out the methodology and I began to be more efficient. So I rarely spend more time on any single project. Yeah, so I don't really see anything that will be too time consuming. Maybe I can tell you a bit about a video that I was making at the beginning of this year. So I wanted to make a 5 minute mini film based on CGI character of the cats in the store with a story. And that took me. Yeah, that actually took me some, some time. Because to make a proper film with proper, a proper story, you need a lot of scenes and a lot of different characters. And the limitation of the AI model is that it can be trained with a lot of data, a lot of image data from everywhere, but it may not cover everything. So some of the scenes will be almost impossible to create. Now you have to change the story. You have to change the story based on what images you can create. That's the time consuming part.

That's a good. Yeah, that's the thing with generative AI, right. So sometimes you think you're going to get something consistent and you know, another video that can work really well with the other videos and then it's going to spit out something completely random that you can't even work with. Right. So. So Tanya, like we, we've talked a lot. We've, we, We've, you know, talked about some of your processes. We, we've showed the audience live, you know, how to kind of roll the dice and work with these different generations. But you know, as we wrap up here, what is your best piece of advice for, for people out there? Maybe they're new to these AI video programs or maybe they're just really struggling to, to get good results. What is, you know, kind of like especially based on, on your methodology, right, of kind of rolling the dice. What's your best piece of, you know, advice for everyone to improve their outputs?

I think there's one shortcut that we can, that we can all follow. Basically for any image model or for any video model, you just look at the successful images that people, that other people create or you look at their marketing video and then based on the marketing video you will know what kind of categories, what image categories, what video categories are easier to create. Then you just double down your efforts into this category. You have a much higher chance of success in if you just focus on the ones that other people already prove that prove their success.

That's, that's great. And you know, I think this is important because this allows. And I guess one more question, right? So I think this allows everyone to, to be more creative, right? Because I think a lot of times when people hear about these tools, mid journey Runway, etc, they're like, okay, well that's not my background. And I think, you know, you just gave a great example, you know, going from someone in market research to now, you know, you're a visual artist. So maybe one last question is what ways might you recommend other people if they find this technology fascinating. But what are just some good ways that people can use this across their businesses or personal projects?

I think the best way is just to get started. Just start with baby steps and start talking to ChatGPT. Start creating images, start with very simple prompts, and then build up your process little by little.

That's great. Yeah, I mean, you have to start small and you have to go step by step. Well, Tianyu, thank you so much for joining the Everyday AI show and telling us all and teaching us all how to roll the dice for better AI visuals. We very much appreciate your time.

You're welcome. Thank you so much for your time.

All right. And hey, as a reminder, y', all, we're going to be sharing some of his best examples. If you want to watch the video, maybe if you're listening on the podcast, make sure to go to your everyday AI.com Sign up for the free daily newsletter. If this episode was valuable, please consider sharing it or sharing it with a friend. Leave us a rating, but we hope to see you back tomorrow and every day for more Everyday AI. Thanks, y'. All.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic. Visit your everydayai.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.