OpenAI’s DALL-E 2: Top 5 New Results! 🤯

Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai. Today we are going to talk about OpenAI's amazing image generator AI DALL E2. And you will see that it can do two things that perhaps not many of you have heard of. So what is Dall E2? This is a diffusion based AI model, which means that it starts out from noise and gradually changes it to become a beautiful image like this one. The image can be anything. So much so that we can enter a piece of text and magically out comes an image that fits this description. It would be hard to overstate how transformative this will be to digital art. Imagine that you could now illustrate any novel. And not only that, but it wouldn't even need any additional work for you to write up the text prompts. Why? Well, just take excerpts from the novel and there we go. And if you feel like adding crazy prompts. As you see, it won't complain about that either. But it can do even more. For instance, if we have an image that we really like, variant generation is also possible. This means that we can provide it an input image. It tries to understand what it depicts and create several versions of it. And it does that spectacularly. But that's not the only little known feature it has. It can also do two other amazing things. One is image inpainting. What is that? Image inpainting means that we take an input image or or video, cut out an undesirable part, and let an algorithm fill in the holes with data that makes sense given the context of this image. Now, previous techniques were already capable of this, but I am very excited to see what Dall E2 can do with this problem. Because we know that it has a really detailed understanding of the world around us. So can it use its knowledge of the world to create high quality inpaintings? Oh boy. You will see in a moment that it can do incredibly well. Let's give it a try. Oh yes, we can delete this squirrel. And here comes the key. Oh my. Remember that I said that inpainting techniques fill in the holes given the context of the image. But Dall E2 does something even better. It fills in the image with any context we can think of. Let's try that. How? Well, this image is ridiculous. Squirrels are not known to be able to operate computers. So let's add a hippo instead. And there we go. Much better. Loving it. Now let's push it to the limits through five outstanding examples. 1. The ladybug. Here's a real photo but look, unfortunately some parts of the photo are out of focus. Oh yes. How about deleting those parts and have them inpainted with Dolly too? Could that work? It knows that this is a ladybug after all. Let's see. Wow, that is much sharper and fits the image really well. I would have to look for quite a while to find out the trickery. What about you? Let me know in the comments below. So this is fantastic. But you know what is even more fantastic? Now hold on to your papers because it can also do not only image inpainting but image out painting. What is that? Well 2. Ever wondered what might be outside of this frame? Well, let's ask Dolly too. And that is insanity. It is really doing it and we can just zoom out and out and out and it feels like it never stops and it just gets crazier over time. If a human would have made this, I would say that human is very creative. But this. This was done by a machine. Is that a hint of creativity in a machine? Food for thought. Anyway, I love this one. What a time to be alive. 3. Of course we need to do that to the Mona Lisa too. What could be outside the frame? Aha. I see. This is a modern take on the old classic. Working at the office while everyone else is enjoying life. Note that once again a prompt was used to generate the theme of the outpainting here. These are the prompts that were used for this example. And these are so good. Let's do a couple more. 4. Zooming out from a house in the middle of a lush field to this very artistic. And it turns out. Wow. That we are on a floating island in the middle of the universe with a twist. And last but not least, five. The last supper. But this time let's not zoom out, but let's try to answer one of history's big questions. And that is what might be going on in the adjacent rooms. Look. I love it. And I wonder what Dall E3 will be capable of. Remember, Dall E2 came out just about a year after the first version and it was leaps and bounds better. And the competition is also coming. Google already has their IMAGEN and party AIs and even that is just scratching the surface. As you see, the pace of progress in AI research is absolutely incredible. These tools are going to transform the world around us as we know it. Especially now that OpenAI is aiming to deploy this AI to no less than 1 million people in the near future. How cool is that? Democratizing art creation for all of us soon. Bravo. So does this get your mind going? Do you have any thoughts as to what Dall E3 could do? Let me know in the comments below. If you're looking for inexpensive cloud GPUs for AI, Lambda now offers the best prices in the world for GPU Cloud compute. No commitments or negotiation required. Just sign up and launch an instance and hold on to your papers. Because with Lambda GPU Cloud you can get on demand a 100 instances for $1.10 per hour versus $4.10 per hour with AWS. That's 73% savings. Did I mention they also offer persistent storage? So join researchers at organizations like Apple, MIT, and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs. Com papers to sign up for one of their amazing GPU instances today. Thanks for watching and for your generous support, and I'll see you next time.