OpenAI DALL-E 2: Top 10 Insane Results! 🤖

Dear fellow scholars, this is Two Minute Papers with Dr. Zsolnai today. I am so excited to show you this look. We are going to have an AI look at 650 million images on the Internet and then ask it to generate the craziest synthetic images I have ever seen. And wait, it gets better. We will also see what this AI thinks is. It looks like. Spoiler alert. It appears to be cuddly. You'll see. So what is all this about? Well, in June 2020, OpenAI created GPT3, a magical AI that could finish your sentences. And among many incredible examples, it could generate website layouts from a written description. This opened the door for a ton of cool applications. But note that all of these applications are built on understanding text. However, no one said that these neural networks can only deal with text information. And sure enough, a few months later, scientists at OpenAI thought that if we can complete text sentences, why not try to complete images too? And thus ImageGPT was born. The problem statement was simple. We give it an incomplete image and we ask the AI to fill in the missing pixels. If we gave it this image, it understood that these birds are likely standing on something. And it even has several ideas as to what that might be. Look, a branch, a stone, or they can even stand in water. And amazingly, even their mirror images are created by the AI. But then scientists at OpenAI thought why not have the user write a text description and get them a really well done image of exactly that? That sounds cool. And it gets even cooler the crazier the ideas we give to it. The name of this technique is a mix of Salvador Dali and Pixar's Wall E. So please meet Dall E. This could do a ton. For instance, this understands styles and rendering techniques. Being a computer graphics person, I am so happy to see that it learned the concept of low polygon count rendering, isometric views, clay objects, and we can even add an X ray view to the owl, kind of. And now, just a year later, would you look at that. Oh wow, here is Dolly too. Oh my. I cannot tell you how excited I am to have a closer look at the results. Let's dive in together. So what can it do? Well, that's not the right question. By the end of this video, I bet you will think that the more appropriate question would be, what can't it do? This one can take descriptions that are so specific, I would say that perhaps even a good human artist might have trouble with. Now hold on to your papers and have a look at 10 of my favorite examples 1. A panda mad scientist mixing sparkling chemicals. Wow, look at that. This is something else. It even has sunglasses for extra street cred. And the reflections of the questionable substance it is researching are also present on its sunglasses. But the mad science doesn't stop there. 2. Teddy bears mixing sparkling chemicals as mad scientists. But at this point we already know that doing this would be too easy for the AI. So let's do it in multiple styles. First, steampunk. Second, 1990s Saturday morning cartoon. And third, digital art. It can pull off all of these. Three. Now about variants. Give me a teddy bear on a skateboard in Times Square. Now this is interesting for multiple reasons. For instance, you see that it can generate a ton of variance. That is fantastic. As a light transport researcher, I cannot resist mentioning how nice of a depth of field effect it is able to make. And would you look at that. It also knows about the highly sought after signature effect of the lights blurred into these beautiful balls in the background. The AI understands these bokeh balls and the fact that it can harness this kind of knowledge is absolutely amazing. 4. And if you think that is too specific, you have seen nothing yet. Check this out. An expressive oil painting of a basketball player dunking depicted as an explosion of a nebula. I love also has a nice space jam quality to it. Well done little AI. So good. 5. You know what I want even more specific and even more ridiculous images. A propaganda poster depicting a cat and dressed as French emperor Napoleon holding a piece of cheese. Now that is way too specific. Nobody can pull that off. There is no way that. Wow. I think we have a winner here. When the next election is coming up, you know what to do. 6. And still, once again, believe it or not, you have seen nothing yet. We can get even more specific. So much so that we can even edit an image that is already done. For instance, if we feel that this image is missing a flamingo, we can request that it is placed there. But we can even specify the location for it. And hmm, even the reflections are created for it. And they are absolutely beautiful. Now note that I think if there are reflections here, then perhaps there should have been reflections here too. A perfect test for one more paper down the line. When Dall E3 arrives, make sure to subscribe and hit the bell icon. You really don't want to miss it. 7. This one puts up a clinic in understanding the world around us. This image is missing something. Missing What? Well, of course corgis and I cannot believe this. If we specify the painting as the location, it will not only have A painterly style, but. But one that already matches the painting on the wall. This is true for the other painting too. This is incredible. I absolutely love it. And last test. Does it? Yes, it does. If we are outside of the painting at the photo part, this good boy becomes photorealistic. Requesting variants is also a possibility here. So what do you think? Which is the best boy? Let me know in the comments below. And number eight, if we can place any object anywhere, this is an excellent tool to perform interior design. We can put a couch wherever we please. And I am already looking forward to inspecting the reflections here. Oh yes, this is very difficult to compute. This is not a matte diffuse object and not even a mirror like specular surface, but a glossy reflection that is somewhere in between. But it gets worse. This is a textured object which also has to be taken into consideration. And proper shadows also have to be generated in a difficult situation where light comes from a ton of different directions. This is a nightmare. And the results are not perfect. But my goodness, if this is not an AI that has a proper understanding of the world around us, I don't know what is absolutely incredible progress in just one year. I cannot believe my eyes. You know what, Number nine? Actually, let's look at how much it has improved since Dall? E one side by side. Oh my. Now there is no contest here. Dall? E 2 is on a completely different level from its first iteration. This is so much better. And once again, such improvement in just a year. What a time to be alive. And what do you think if Dall? E3 appears, what will it be capable of? What would you use? This? Or Dolly 3? 4? Please let me know in the comments below. I'd love to know what you think now. Of course not even Dall? E 2 is perfect. Look at that. Number 10. Inspect the pictures and tell me what you think. The prompt for this must have been not easy, right? Let me know your tips in the comments below. Well, it was a sign that says deep learning. Well, a for effort, little AI. But this is certainly one of the failure cases. And you know what? I cannot resist. Plus one, if you have been holding on to your papers, now squeeze that paper at least as hard as these scholars are squeezing their papers. So which one are you? Which one resembles your reaction to this paper? The best? Let me know in the comments below. And yes, as promised, here is what it thinks of itself. It is very soft and cuddly. Or at least it wants us to think that it is so food for thought. And if you speak robot and have any idea what this writing could mean, make sure to let me know below. And one more thing, we noted that this AI was trained on 650 million images and uses 3.5 billion parameters. These are not rookie numbers by any stretch of the imagination. However, I am hoping that with this there will be a chance that other independent groups will also be able to train and use their own Dall E2 and just in the last few years OpenAI has given us legendary papers. For instance, an AI that can play hide and seek, solve math tests, or play a game called Dota 2 on a World champion level. Given these, I hereby appoint Dolly into the pantheon of these legendary works and I have to say I am super excited to see what they come up with next. This episode has been supported by Lambda GPU Cloud. If you're looking for inexpensive cloud GPUs for AI, check out Lambda GPU Cloud. Get this they've recently launched an Nvidia RTX A6000 with 48 gigabytes of memory. And hold on to your papers because Lambda GPU Cloud can cost less than half of AWS and Azure. Join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today. Our thanks to Lamda for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.