Dr. Zsolnai covers NVIDIA's Perfusion, a technique that lets text-to-image AI learn custom objects like teddy bears or teapots from a few examples. A user study found personalized outputs matched the quality of original images, and the method uses rank-one edits to keep the network efficient.
Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to have a look at an image creation AI called Perfusion that is completely out of this world. Imagine a text to image AI like stable diffusion or midjourney as a chef that can cook any dish. However, sometimes we like the dish we get, but we would like to introduce our own ingredients to it. And. And normally if we write a new recipe, a new text prompt, we get a completely new image. A new dish. However, that is not what we want. What we want is the previous dish with a new ingredient. Let's have a look at an example. We have photos of our favorite teddy bear and we would like it to create an image of it playing with a ball in the water. A previous technique does. Uh oh. I have already looked at this image and I have bad news. It's not going to be pretty. Are you ready? Ok. Here you go. Whoa. It seems to have fallen apart. And there is no ball anywhere to be seen. Now let's look at the new technique. Now that's what I call a great result. Love it. However, this is an emerging research field, so of course this is not the only previous technique out there. Let's take this adorable cat. Yes, this cat is missing a pair of headphones. So let's add it with another previous technique. And nothing.
Look.
At least I see the remnants of some crazy thoughts this earlier AI has about headphones. It sees them more as yo yos. Now the new technique. Oh yes. Fantastic. And of course the new technique knows that you can never wear enough headphones. So there we go. So cool. So this new paper finally enables this kind of personalization with something that they call concepts. What is that? That is the AI Chef's cookbook. Having a teddy bear dressed like a wizard can be a concept. A sombrero can be a concept. All of these can be concepts. And here come four incredible kickers. 1. As many of you fellow scholars know, some people like pineapple on their pizzas, and some people like eggs on their pizzas. Well, here is the kicker. It can fuse concepts together as well. This is an AI that can make that pizza happen with pineapple and eggs at the same time, but without sacrificing the quality of the food. Now that sounds like a huge challenge. Let's have a look. First, take a teddy bear and a teapot. Then ask this little AI to dress up the teddy bear and upgrade the teapot to have a cartoonish look. And now comes the magic Let concept number one. Teddy bear fuse with Concept number two, the teapot. And there we go. Wow. And that is not any teapot. That is exactly that teapot. The same objects can be reused. With this new paper, we can not only perform text to image, but we can build these little worlds exactly the way we want. This AI chef feels like it can cook anything. We can keep our characters and add our personal touch to them without losing this character. They also provided a a little taste test, a user study where a bunch of humans were asked which examples they liked best. And it came out that they not only liked this new perfusion technique the most. I have to say I expected that at this point, but they also found something that I did not expect at all. Get this. The new personalized images were found to be almost indistinguishable in in terms of quality from the original. Wow. And finally, number four, all this is done through rank one edits to the neural network. What is that? That is a proper surgical approach. This means that it introduces minimal changes to the neural activations of the neural network to achieve all of these changes. And that is likely why we are not losing out on the quality of the image. This is an incredible feat. Bravo, Nvidia.
If you're looking for inexpensive cloud GPUs for AI, Lambda now offers the best prices in the world for GPU cloud compute. No commitments or negotiation required. Just sign up and launch an instance. And hold on to your papers. Because with the Lambda GPU cloud, you can now get on demand H100 instances for just 1.99per hour and yes, $199. And they are one of the first cloud providers to offer publicly available on demand H100 access. Did I mention they also offer persistent storage? So join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today.
Thanks for watching and for your generous support and I'll see you next time.
