The Pixel Revolution Part 2 with Suhail Doshi, Founder of Playground AI

I try to sometimes put myself in the shoes of, you know, let's say the artists or the people making these images, photographers, whoever. We were the first site ever, and I think the only site where if there was a prompt on our site and someone references his name, we directly link back to his page. It might be generally okay to make things. In fact, many brands are perfectly fine with fan art or whatever. There's kind of this question of commercial use. That's where things kind of like stop. If this whole thing is very gradual, then I think Probably society will find some way to assimilate to it. If it's vastly faster than that, then I think that we definitely need to do something about that. I think that who it's impacting very much needs to be considered. Some craft that they've been doing for a decade or two, you can't ask people to evolve and upskill.

Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs, and builders working on the frontier of artificial intelligence. Each week we'll explore their revolutionary ideas, and together we'll build a picture of how AI technology will transform work, life, and society in the coming years. I'm Nathan Labenz, joined by my co-host, Erik Thornberg. Hello and welcome back to The Cognitive Revolution. My guest today is Suhail Doshi, founder and CEO of Playground AI. This is a special episode for me because Suhail was our very first guest on the show almost a year ago, and today he returns as roughly our 100th guest. Of course, it's been a busy year for everyone in AI, and Suhail and team have been hard at work building and releasing a huge number of new features designed to help anyone create and edit images like a pro. Most recently, they've trained and open-sourced a new foundation image generation model called Playground V2, which is preferred to Stable Diffusion XL some 70% of the time while being built on the same exact architecture. A decision that Suhail made so that the open source community can easily adapt and apply all the surrounding tools that they've already developed. If you're wondering how it makes business sense for a company like Playground to give the fruits of such a major investment away for free, Suhail's perspective on the state of AI art in general and on image generation in particular might surprise you. Because yes, of course, it's undeniable that the state of the art has continued to advance. And today, not only Midjourney and DALL-E 3, but also Stable Diffusion XL and Playground V2 can generate excellent quality, nearly photorealistic images with increasingly fine-grained prompting controls. And yet in Suhail's analysis, we're still only at roughly the GPT-2 stage of image generation AI development, just scratching the surface of all the promise that AI-based image manipulation still holds. Most of the use cases, as Suhail has come to understand them in talking to Playground users, are not very well served by existing models, and the leading AI artists use a mix of models and different complementary tools to achieve their best results, meaning that things overall are still a bit too complicated for everyday casual users. So what's missing from this picture? A unified vision model that can do it all— understanding, generating, and also manipulating images in all sorts of useful discrete ways. This does not yet exist, and the leading large language model developers don't seem to be really focused on it. So Suhail and team are setting out to build it over the course of 2024. As always, if you're finding value in the show, we'd ask that you take a moment to share it with friends. It has been incredible to see how the show has traveled and how the audience has grown entirely through word of mouth. And I think this episode would be perfect for both the artists and the application developers in your life. Now, I hope you enjoy this conversation about the state and future of AI vision and image generation with Suhail Doshi of Playground AI.

Suhail Doshi, founder and CEO of Playground AI, online at playgroundai.com. Welcome back to The Cognitive Revolution.

Thanks for having me, Nathan.

Yeah, I'm excited. So you were our very first guest when we launched the show almost a year ago now, and I think you'll be Roughly our 100th guest. So this is a cool way to, to celebrate having put out a lot of episodes and another trip around the sun. And I'm excited to catch up with you on everything that's been going on in the world of pixels, both broadly and, and at your company over the last year. For starters, I just listened to another interview that you did with Swyx and Alessio on the Latent Space podcast. I thought that was really cool.

Yeah.

good and figured we'd try to cover, of course, there'll be some overlap, but try to cover largely some different topics today. I guess for starters, I'd love to hear kind of, you know, what's new at Playground, what's new in image generation. Yeah, I'm sure there's a lot and I'll, I'll, I'll have some follow-ups, but I would love to hear how you would kind of summarize the journey over the last year.

Yeah, I think the journey over the last year has had like some interesting highlights and some interesting kind of like lowlights. I'll start with the lowlights because Even though it may sound slightly depressing, I actually think it's the most exciting bit. Of course, everybody knows that things like DALL-E 3 have come out, Midjourney V6 has come out, Stability AI released SDXL back in, I want to say, June, July. So there's been a lot of new foundation models that have come out, but I would mostly say that I have been fairly disappointed. By the progress. And one way that I would maybe articulate this is that mostly the models are used for art, and art has some value and utility in the world, but for the most part, we haven't really seen that general model high utility that you get out of these language models. And so to kind of just hearken back a little bit, maybe 3 or 4 years ago when we looked at language, we sort of said, what are these What do these AI models really do? They might summarize something, hallucinate. Maybe they have sentiment analysis was kind of like a use case that was touted a lot. Many startups were built just doing sentiment analysis, those sorts of things. Like the models couldn't rhyme, the models couldn't really write code, the models couldn't really do anything. So there was like these very specific use cases. And then as folks at either Anthropic or OpenAI noted, with scaling laws that they felt like maybe the models could get better and more generally useful. And the only reason why I'm taking us through this hopefully well-charted history that people know that are in the AI world, but if you're not in the AI world, that's a rough snapshot of what happened with text. And now you have things like ChatGPT and it's writing code and you can ask it questions. You don't have to go to Stack Overflow anymore and whatnot. That really hasn't happened with vision at all. You know, what we got out of vision was I can make really amazing sort of like art. I can make a meme, I can make something cool, I can show it to my friend. Maybe I can make like a book cover. Maybe I can make, you know, one use case for some of these models is making coloring books.

Sure.

But the models, and the models are really great at extrapolating like interesting characters or subjects or environments. And making art, but they're not very good at kind of doing anything else. And they're not good at manipulating pixels in any other kind of way. And so I actually was maybe expecting last year that perhaps like the pace and momentum would be a lot faster, but it just turned out that wasn't the case. And so maybe the most, the thing that I'm most excited about is that it kind of feels like vision is like the year. like a year or 2 off from this like big moment that language has had. Um, and there's far more huge quantities of vision data compared to text.

Yeah, no doubt about that. There's plenty of, my dad has this, uh, crazy dad joke of, you know, we're running out of pixels and the pixel mines are getting depleted. And the reality is obviously there's, there's plenty of pixels flying around if you can figure out how to use them. That is an interesting perspective. Would you, This is, it's maybe so strange as to not be useful, but would you say we're at like a GPT-2 level, you know, relative to some inflection point that you're expecting for?

Yeah, it's funny that you asked that question because that is exactly how I phrase it at our own company. Or if I'm talking to people that we're trying to hire, I'm kind of like, you know, I'm not even sure we're at even GPT-3 level, even though there's this feeling maybe last year that perhaps maybe we were. But I actually don't think, I don't even think, uh, sorry, that we were at like something greater than a GPT-2. And I think the continuum of, you know, where we were 2, 3 years ago with GANs and stuff, things have significantly improved, but the overall utility hasn't really significantly improved. You know, we've gotten our kind of like, like text, we have our sort of 3 or 4 very simple use cases. You know, you can make art, you can make, um, you can remove something, an object, you know, with the pixel phones removing things. You can remove a background and that's about it. That's about roughly what you can do with images. Whereas in the domain of text, it's like the long tail of what you feel like you can do, the value that you get is just so much more significant. And so I say that this is mildly depressing in some ways, but I'm very excited and I'm working on it because I feel very excited about the possibilities that actually it's this huge open field of unsolved problems. And now there's a lot of momentum, a lot of effort, and a lot of desire to make it better.

Yeah, that's really interesting. I mean, I think for a lot of people that probably even who are in the AI space pretty deeply, I think that would probably come as a bit of a surprise. You know, as you were talking, like, I, I have remarkably little information about who the Cognitive Revolution audience is, but we're definitely not like an AI art or, you know, image generation or like a pixel specialist. show. So I would imagine most people have a lot more experience with your ChatGPTs and your Claude's as opposed to with image creation. But from the outside, and I use these tools, like, I'm not an artist, but I use them periodically. It definitely feels like things have gotten a lot better, right? I rewind to a couple years ago when I was seeing this stuff start to bubble up on Twitter, And seeing Rivers Have Wings, the OG true legend accounts. Rivers Have Wings, if you're listening, check your DMs. I want to have you on the show. But that stuff was pretty gnarly and it was like, wow, it's amazing that you can do anything. But it definitely was not passing any sort of visual Turing test equivalent or whatever.

It was sort of like swirling colors and backgrounds and abstract shapes. And now we're getting kind of this Amazing photorealistic thing, kind of solved hands. We're starting to see models with like text, text synthesis and that kind of thing. So it's definitely, definitely on an amazing pace. It's definitely not like hitting some odd, weird asymptote quite yet.

Given that progress, you said utility hasn't grown as much as you would have hoped. Is it the case that we maybe naively misunderstand or kind of misconceive of what actually drives utility? Like the fact that we're able to generate these Photorealistic outputs today is just not enough for a lot of the utility that you're looking for?

I generally think for the most part, it's just at the moment mostly a lack of imagination. It is somewhat hard to imagine extrapolation from these big models. I think even at GPT-3, it wasn't like there was this massive audience thinking that. I think if you had gone back in time to maybe when GPT-3 came out, maybe 6 months later, I don't know that there was like this very, very big audience that thought for sure, like what we were getting out of GPT-4 was coming. I don't think that I thought at maybe even kind of like last year, I had access to GPT-4 maybe in, I want to say like October of the year before last. And I'm not— the first thing I did with it was I wanted to see if it could find vulnerabilities in my computer programs. I was like, how good would this be at finding security issues? That was the use case I had in mind. But I'm not sure that I fully internalized even back that year that this thing would be so instrumental in writing code as good as it is. And I wasn't sure that I thought that it could answer relatively difficult questions about my life or things happening in my life, or if I have a leak in a ceiling, how might I solve that? I don't think people thought that it could be— not very many people thought that it could be that powerful. And so I think that's just similarly true in images. You're just sort of like, okay, it can make this beautiful art. It's really fun and novel, but what do I use this for day to day? Because you actually look at images, it kind of struggles making logos. It sort of struggles with spatial reasoning. It's certainly like things like inpainting or editing are extremely difficult. We're still sort of talking like eyes are still sort of like a bit strange. Hands aren't always coherent. Sometimes it adds kind of like an extra letter if you're asking for text. So there are like these little things that we see are present in the models. But there are some, I think there are like much bigger things about vision just generally that are probably like can lead to like a very big general model. Like one of the things that we tend to think about at Playground Is how we might achieve a unified vision model, right? And there's like 3 pieces to at least graphics, which is just a subcomponent of vision. There's creating pixels, there's editing pixels, and then there's understanding them. So GPT-4B is like a good example of like a very rudimentary model that's like getting better at understanding things. Right now it's quite good at like captioning, but there's like other reasoning capabilities. provision, um, or just like understanding an image or even a video. Like, it can't do video, for example. Right now we're kind of stuck on creating, but we're not very good at like editing things. Like, one thing that hasn't really happened yet is there hasn't been a very big effort on manipulating real pixels of real images— or sorry, pixels of real images, right? You're mostly dealing with synthetic images, but we're not doing a whole lot with, um, real photographs where I think there's probably a lot more utility and a lot more value. I'll give you an example. If my son was smiling for a Christmas card that we wanted to send out late last year, it's really hard to get the dog and the kid and everybody in the picture, getting them to smile appropriately. But it'd be sure, it would be nice if I could just be like, hey, highlight his face and just say, can you make him smile? Or take 2 images, one where he's smiling, one where I'm not, and finding a way to merge those 2 things together. That's a very simple application. And so you can imagine What the other kinds of general useful things that you could do with manipulating pixels. And that's maybe the tip of the very, very tip of the iceberg. So I just imagine that there's a lot more for vision. Understanding the context of a video scene is another example. If you asked a model to look at 3 minutes of a video and what happened, could the model reason about that? So just a bunch of those kinds of things. And so there's just this general lack of I would say significant investment with regard to trying to make a great general vision model. Hey, we'll continue our interview in a moment after a word from our sponsors.

Yeah, I would love to hear a little bit more about the use cases. And also I'm curious about what shortcomings you are finding in GPT-4V. I have used it pretty successfully, actually, at least for the use cases that I've tried. One That was like a kind of mind-blowing moment was I took my kids and I can definitely relate to the difficulty of getting them all to smile in a photo because we're up to 3 and, you know, 2 out of 3 feels like the best we could ever manage to do. But I took them to Salem, Massachusetts leading up to Halloween because we were out there for an event that my wife was organizing. And it was actually a, you know, Fun little day, but I came across this parking sign and this had just at the time there was the famous one on Twitter where somebody was like, I'm never getting a parking ticket again because he took the picture of you know the eight different parking signs and said, "Can I park here right now?" My version of that was the Salem version where it had some message about like you know special parking rules for October and at the bottom it said Salem PD and I was like, "What's going on here?" And it was able from the Salem PD and the like. special rules for October to infer that it appears that you're in Salem, Massachusetts, which has this historical connection to witchcraft and whatever. And now it's like a holiday destination leading up to Halloween. And so it's probably that that you're dealing with. And I was like, man, this thing, there's a lot of knowledge in there that it's kind of able to tap into. And we've had really good luck with that at Waymark too, where we just use it to filter user images. We've long had this product experience where we'll pull in a ton of images, like every image on your website, if you're a small business, just for your convenience. So you don't have to go load them into our product manually. So we kind of pull them in, but then we get all kinds of crap, right? So how do we filter that to use the relevant stuff? And GPT-4V is awesome at that for us. Literally just saying like, here's the profile of the business, you know, which of these images would be good to use, which, which would not be good to use. I've done a little bit with breaking video into frames and trying to get it to understand that as well. I wouldn't say I've like fully characterized how well that can do, but anyway, those have been my experiences. Have you, what things have you seen or done that you feel like, you know, leave something to be desired? in the GPT-4V performance?

I think one small, easy one is that GPT-4V is not always great with certain tasks like image segmentation. It can be kind of weak at that occasionally. Image segmentation or finding the right bounding box for certain images tends to be extremely useful. There are just better models than GPT-4V. Arguably a lot faster. Segment Anything is a really great example. You can use Dyno plus Segment Anything together to achieve really interesting state-of-the-art results. That's a very simple use case that's been going on for a long time in AI that probably could greatly benefit from a general model. I think the promise of something like 4B is really outstanding. That task is so valuable for a whole bunch of use cases related to Camera, anything related to cameras is a really good example of that. Sometimes I still think there are some issues with hallucinations with its descriptions of some of the images, but I love 4B. My favorite use case is taking screenshots on my desktop computer and then asking it to solve some problem of mine. Lately, I've been just doing this funny thing where I'll just take screenshots of errors in my code and asking it to Basically help me fix whatever the problem is. It's almost like I can be really lazy about things. But I had been thinking that how valuable it would be to sort of like have— it surely would be valuable if there was a model that was able to just like stare at my computer with me and kind of help me along as I go. And I hope someone at OpenAI is sort of working on something that can be like, hey, I might be able to help you with that. I'm kind of pairing with you. As you're working. But I think that's going to take like a lot, a lot of work. And that's just, that's just one domain, I think, of like pixels and images. You know, it's just, that's just understanding what it's seeing. Um, that's not even necessarily, um, manipulating graphics in any kind of way.

Yeah, certainly the fact that it is all text out is a major limiting factor in terms of what it can do with images.

Because all I do is spend my time on vision. I spend a lot of time about what are the major benefits of vision versus say language, not just in the training data, but also in the outputs. And I'll give you another example of something. One thing that's really challenging with text is it takes a lot of effort to read text. It takes a lot of patience. It takes a lot of focus. And we continue to live in a world where there's even less and less of that. But one thing that vision is, graphics or vision is really good at is just a very amazing visual explanation of something. Often there are circumstances where you want a visual explanation of something. We see this all around us. We know when we read a book and we see a diagram or something that it can be very valuable, or a graph. A graph can be extremely valuable. But there are other modalities like a video that's explaining something to me is really valuable. Audio is obviously valuable, but it tends to— there are major areas where Things tend to lack in that regard. For example, there are a lot of people that are just kind of waiting around for these image models to figure out how to make diagrams, flowcharts, diagrams, ways to be able to understand something about the text. That'd be one thing. We all can very much tell when something doesn't look like us. One thing we learned over the last year is that humans are really good at looking at faces. It can really tell when something is off. You might have 10 friends that tell you that That cartoon version of you looks like you, but sometimes when you look at it, you're like, that doesn't look like me. You're almost offended by it. And so I think that there's all these different kinds of use cases that are just very unsatisfied. But I mostly think that language doesn't accommodate as well as it could, and it will never accommodate. And that's totally reasonable.

Yeah, those are 2 good examples. You know, I've tried to work around the graphic thing a little bit by asking GPT-4 to create a text-based, you know, basically to follow, there's like a bunch of, as you're probably aware, there's a bunch of different flowchart and diagram kind of syntaxes that you can use. So I've asked it to create some diagrams that way. And there's one called Mermaid and there's a, there's a, you know, VisGraph, I think, is another one that's worked okay. But yeah, definitely. I did this for Waymark, actually, with the AI system that has a bunch of different models that even for us on the development team, it's sometimes a little bit hard to remember, like, wait, which has to finish before the next thing? What are we parallelizing and what's the dependency on what? So to just try to have that at a glance is something that didn't have. And I was able to get a pretty decent version of it from this syntax that then could render deterministically into the visual form. But you're definitely right that it leaves a ton to the imagination compared to what a real infographic or proper designer would do.

It's nowhere close.

Yeah, that's one that people talk about. A less common one could be as simple as something like, Here's a picture of my house, my room in my house, my bedroom. I need to figure out a way— I want to figure out different combinations. If I put my bed on this side of the wall, what should my room look like? If I put my bed on the other side of the wall, keeping all the same objects in place, how would you reorganize my bedroom?

Right.

And if you could copy and paste that, you could put it in GPT-4B and it will spit something out. Right. But it's very hard to imagine what— and you have to now read this thing. It's extremely hard to reimagine your bedroom. But imagine if there were an image model that could completely reorganize your bedroom and show you all the different ways that it could work, right? Imagine that you could iterate with it. You could say, hey, what if the dresser was made out of this other material? What if it was a little bit bigger? What if the nightstand was lower? There are a whole bunch of visions. So I think if you can try to imagine these little simple things that feel very hard today, it's very hard. Where do I go? What website do I go to? Okay, I got to start drawing lines and stuff. Ah, forget it. I don't want to do it. Too much work. So if you can try to imagine these circumstances where you're iterating just like you are with language, But with something very visual, whether it's your home or something else, I think you start to get more out of it. You could do the same thing with logos. You could say, I want my logo to be a little like this. Can you get rid of the little swirl? Could you change it to circles, to rounded rectangles? You're able to actually work with an artifact together. That's virtually impossible to do. That's another example of something that would be very hard to do with I think that there are even bigger use cases in vision, whether it's something understanding that it's your face versus an intruder in a camera system, something that learns that over time. There is a whole limit to what we probably can do with language that I think is underappreciated with vision. And I think it's just strictly because we're trapped in our world right now. And I think that's going to change very significantly this and next year.

I have kind of 2 lines of thought going in my head at the same time. One is, how's that going to happen? You guys have just trained a foundation model from scratch yourselves. I'd be interested to hear kind of what you think the trajectory of foundation models in vision is going to be. And then I'm also really interested in, and I think it's a very general problem, although you have a particular form of it, for product owners, how do you build in a way that balances the now and what people need to get utility from your product today versus what is going to be needed or maybe no longer needed as the models themselves get better. I went into the product and I was making some stuff in preparation for this, and I have done it more than once between our episodes, but definitely was just catching up in the last couple of days. I see that there's a lot of new features. I was going to ask, What are some of your favorite new features? What are people getting the most utility from? And I still want to ask that, but I also now kind of want to ask, how are you thinking about what features to build that are like patches for sort of model weaknesses that may not be needed in the future versus what things do you think are kind of always going to be non-model features? So there's a lot there. I'll shut up and you can take it all apart.

No, that's great. I think I think the last year has kind of made me see some of like the community has done such an amazing effort at fine-tuning models, finding fixes, being extremely clever. And it's exactly kind of like how you describe it. It's like, for as amazing of these feats as they are, it is a lot of patchwork. It's not a true fix at a foundation model level.

Yeah.

There are models that fix hands, fix eyes. There are models that do upscaling, try to get more detail out of them. There's tons of amazing tools, but they're all just patches. I think maybe the more surprising thing is we've been thinking a lot about what comes after just text-to-image. What else is left? How do you get to a much deeper understanding of these kinds of concepts. And so I think for the most part, we sort of think of it as these 3 pieces around, okay, how do we create something from nothing? How do we take existing pixels and manipulate them at a really high fidelity? And how do we understand the images, kind of like GPT-4B? And then how do we bring the 3 together? Because ideally that model understands a lot more. And it's far less patchwork and it's far more general than what we have today. We're planning to be very focused this year on editing. So I think the first kind of thing is that I think the community and everybody involved, all the researchers involved, made this wonderful way of creating amazing art. Sometimes it's photorealistic, sometimes it's just fun, entertaining, fantasy art, what have you. And I think that has driven some utility, but definitely not enough. I think the next thing is probably our kind of like not so secret plan that I'm happy to reveal is just that we're going to work on multitask editing. And multitask editing is just, you know, kind of taking existing images and manipulating those pixels such that we can achieve anything. You know, there have been some kind of early versions of this that happened last year. You know, it should be anything from like, what would I look like if I were aged up, you know, at 65 years old? Or my son? What would I look like with a different hairstyle? There are models out there from researchers that are trying to replace clothes in a really high fidelity way. That should really be encompassed in a general model as well. Being able to do all kinds of things like that. Being able to manipulate some kind of interior design would be a good example of that. And then it should be more difficult things. So those would be, some of those would be local edits. And then there's more global edits.

Yeah.

Global edits would be like, you have a big scene that's clearly like a wintry scene that you might want to make like a spring or summer scene. And that means every object, everything in that scene has to be changed. So that'd be a big global edit. I think that we need to get better and better at those kinds of things because I think the utility will rise across customers who want to do more valuable things. And I think a lot of those things tend to be with real imagery. And then I think we're also going to work on a model that's trying to understand everything that's going on in an image. We need good understanding because that actually helps us train the other 2 models. And so hopefully by the end of the year, we're looking at something where we can try to unify these 3 things into a single model that can do more surprising things, more surprising use cases. Just strictly for 2D images. You know, it's like we've kind of moved on to like video and people have moved on to video and stuff. And I think that's really wonderful, makes really amazing art. But I think that we still haven't quite like, we kind of haven't shown how big things could be, even just an image, which is even on its own a hard modality. Hey, we'll continue our interview in a moment after a word from our sponsors.

So how do you think the experience changes over time? Like today, Again, I go in there, I can do the classical text-to-image prompt and get something out. Now I can also bring my own image and ask for modifications that are kind of holistic. Like I, you know, my, one of my early semi-viral tweets from last year was taking the ultrasound image of our now 9-month-old baby that was still in my wife's belly at the time. And, you know, saying, what would this look like if it was a newborn? And it did a reasonably good job of that. I wouldn't say it was, you know, it was still in the uncanny valley, but it was like, you know, good enough that people were intrigued to see it. And you got a lot more features too, right? Now I can outpaint, I can mask and inpaint, I can start with like a line sketch, I can start with a depth map, I can start with, you know, probably a couple other things as well. You have kind of pre-rendering sort of features. There's like the intensity of the guidance. So there's all these kind of knobs and dials that have been added. Do some of those go away? Do we continue to add like all these sort of control elements or does it become more of a like just conversational UI again?

I think some of the elements go away, right? I think some of the elements go away because the model should have an understanding of depth. The model should have an understanding of edge lines. The model should have an understanding of color, lighting, right? The models shouldn't, shouldn't, you know, ControlNet, For example, for these models is really great innovative thing that came out last year, but the model should really understand that. You should be able to erase a part of an image and the model should understand that like actually there is like a big depth component here and that I have to consider the shadows, right? Consider what's in the foreground and the lighting related to a character or subject, right? So I think definitely some of these things go away. And I think that, you know, and for the most part, you know, we find that There are definitely a whole bunch of power-savvy users that can use these tools, but a vast majority of people definitely struggle. Definitely, it takes some time and expertise and watching a couple of YouTube tutorials perhaps to understand these things. So I think those things kind of go away. I don't think it ends up being just text. I think text is great. Text is a really great kind of abstract way that humans can compress a very high-dimensional concept in their mind. to something simpler that they can input into a model. It's very hard to imagine something. How would you take something that you're imagining and then giving it to the model? Well, maybe the best answer for that would be another image. But one of my favorite examples is the concept of shattered glass. Shattered glass is like, we all have a version of what we can imagine that looks like, but actually it's very difficult To both have the same concept of what that means, because there's so much entropy with shattered glass that you and I would have completely different imagination of how that would look and how the lighting would be and what color things would be like. But maybe you gave the model something that you like. Maybe you give it a style. Even style is really hard to describe to these things. Style can be a combination of different things that you like together. If you're thinking about your brand, you might be inspired by multiple things and wish that something could look like all 3 of those things. So I think my sense is that it's probably going to be a combination of language, which gives you some sense of that, but it's going to need something that's even higher dimensionality than that so that it gets closer to what you want. A model that probably emits something visual probably should be very visual. It probably should be Maybe it's using your finger on your phone and masking something. Certainly it's going to be typing something. Certainly it will be like, I like this thing out there in the world. My guess is that vision models will probably have to be extremely multimodal compared to maybe the starting point of language.

When you said the bit about language and just the way that it allows us to compress what's in our head, brain jumped to, well, what about just direct brain reading? I wonder if there is a, because we've seen some pretty interesting results over the last year of, and, you know, with still like fairly cumbersome hardware, but less cumbersome, you know, gradually over time. Do you envision a future where somebody sits down and puts like the Playground crown on their head and sort of like doesn't have to say a word, but can instead Kind of like the, that classic scene from Back to the Future, you know, suction cup the thing to your head and then just kind of focus on your own, like mental state, you know, kind of commune with the system and, and get your ideas out that way. I would have thought that was insane to ask, by the way, even a year ago, but it feels less insane today somehow.

Yeah. I mean, we can get, we can first get words out of, out of, out of our brain. We can go from thought to word. Uh, that would be amazing. I don't know if we can do that. I actually haven't studied anything about brain, trying to get like brain output to get to like any kind of accuracy. I think I saw something maybe where like, maybe, maybe you can like move like an object. Like there's ways to like move left, right, up, down or something like that. But I haven't seen, I don't know if there's any research that's like thought to text.

There's thought to text. There is also thought to image reconstruction. It doesn't necessarily work as well as it might need to work. But this is part of why I, you know, kind of have this AI scouting concept here. I'll give you 2 links and you can check these out at your, at your convenience. But one, the first one we had a guy named Tanishq.

Yeah, I know Tanishq.

What was really interesting about his was that it was low sample was required and they used existing foundation models and kind of figured out a way to with a relatively small per-user sample, map the brain activity into the kind of latent space necessary to then diffuse into the image. But the reconstructions are really quite good. That one is an fMRI, if I recall correctly, which is far from consumer. And then this other one from Facebook or Meta is magnetoencephalography, Which is definitely still cumbersome, but less so and also fast. This one, you can kind of see the accuracy is lower, but if you can reconstruct what they're actually seeing, then if they close their eyes and just imagine stuff, you would imagine that could start to work as well. And I also do wonder just about the degree to which people, a big obsession of mine in general is kind of how the world begins to change in response to AI. And I definitely think that People can train their own minds to work with these systems. And these early results don't have any of that going on yet.

Yeah. I think the ultimate multimodal model, at the end of the day, it's not really about tokens and language. The vision transformers are just taking tokens of an image. They're trying to create some kind of codebook of images by taking patches of an image. And then the language stuff is all subword stuff. But it just seems like at some level, it's just going to be something very byte level. And if you believe that it would just become byte level or something lower, something lower level than these 2 things, then it starts to not matter so much, you know, whether it's like language or vision, then it becomes something else. That then it's like the models have— it's almost like you've given the model like a representation that's like very normalized across all of these different modalities to get to some kind of unification of them. And I do wonder, I do wonder where it's headed. You know, I don't know, I don't know right now. I don't necessarily know that like actually maybe, maybe the transformer is not the right thing, or certainly not tokens of language are necessarily the right thing. Um, it's probably too early to tell because Maybe hardware is not even good enough yet. Maybe the compute performance is not good enough yet. There's still a ways to go to scale text or vision. My first thought with the brain thing was I was like, I wonder if it would be too high dimensionality. I wonder if it'd just be such a— vision is already a lot of input data. So people tend to need to find a way to compress it down to something simpler or encode it rather. So my first thought was I was like, I wonder if the brain has very, very high dimensionality because I don't know anything about this. I suppose you could encode it too. But yeah, it certainly would be cool if you could look at your room and just imagine what if it was there and somehow it knew that you could do that. Perhaps text to whatever is maybe not the ultimate first input. Maybe it is thoughts or something like that.

Well, that's what I see.

If you can do thought to text, Then that's like the right primitive, right? Then you can get from, you know, you can get to text to, uh, maybe anything, whether it's like audio or images or, uh, other text. Then you'd have the right maybe like way to decode, like go from thoughts to like whatever you want. So yeah, I wonder if anyone is working on thought to text.

There are some thought to text ones as well. I wouldn't say, you know, they're kind of in the same general ballpark as the, the thought to image where it's like, Whoa, that's striking. And it is, they do it a very similar way. Like you read a sentence or a paragraph and as you're kind of processing that language, they're kind of trying to reconstruct what you have been processing. And they do get like a decent reconstruction. It's not perfect in the same way that these image reconstructions are not perfect, but they're definitely directionally right. You look at it and you're like, you're not guessing. That's for sure. You know, the resolution is not there, but the, that, that they are onto something is like pretty clearly true.

I'd love to spend more time on brain interfaces. I have no idea.

When in doubt, go back and, you know, see what Kurzweil said in the '90s. And then, you know, we're like getting to the time when, when it was supposed to happen. And, you know, that guy has been, I've started to use the term Kurzweil's revenge because I, Came across his work in the mid-2000s. At that time, it was like the exponential curve that he was projecting, we were still in the low part. And so it was like, he said this 5 years ago, but not much was supposed to happen in these 5 years. So whatever, maybe it'll happen. Then 10 years went by and it was like 2015 and it was like, wow, he was wrong about everything. And this guy, what a dreamer. And then now here we are. It's like, actually, exponentials are crazy. He's maybe a lot more Right than not. Apparently he's got a new book coming and it's The Singularity Is Nearer or something. I'm not sure if that was the joke title or the real title.

For what it's worth, most of the time when I talk to people about a unified vision model, even researchers in the field have often struggled, have often not thought about it. A lot of people are sort of trapped in this kind of like, we're just trying to do text-to-image. And they haven't really thought too much deeper about that. Or the other version of this is like a very narrow use case, right? Like e-commerce product placement or swap, you know, changing clothes or filters like on Snapchat, that kind of computer vision stuff. But that kind of computer vision stuff was very, you know, 2020, 2019-style AI, right? That was pre-generative AI, pre kind of like big generative models that can— I mean, the promise of these generative models, foundation models specifically, is that they generalize so well to these other kinds of things that we're surprised by them every day that we use them. We're like excited about GPT-5 because we're not— because not— and we don't even know what it will do, but because we're surprised by what we're We're excited by what we'll be surprised about. And I think that that same moment just hasn't happened with vision. So I feel like I've kind of caught you a little bit off guard during this chat, but I just wanted to mention to you that most of the time when I'm talking to people with vision, they don't see it either. Oh, there are a few people who've hired those people that happen to, that have kind of the ambition or goal to try to do something a lot bigger But we do not have a general vision model of any sorts. Um, and you can tell, you can tell by it, by its limited utility. Maybe, maybe it will be clear towards the end of the year when we go, whoa, I didn't know that we could do that.

Yeah, I think that is a good point of comparison. You know, I would agree that the sort of surprising capability doesn't seem to have happened as much on the image side as it has on the Language side. So it's a good contrast.

But just consider if everyone at OpenAI— and so there's this wonderful, huge language race. It's great. I love it. I want people to focus on that because we're focused on vision. But consider for a moment if all OpenAI was doing was vision. If we could imagine that there was actually a version of OpenAI that was super laser-focused on vision, And right now there isn't, as far as I know. There might be, but I don't think there is. What would these models look like? I think they would be far more superior than DALL-E 3, vastly better than that.

I guess one interesting question is what does and doesn't exist in the world that is kind of— in text, everything-ish exists, right? You may not have access to all of it, but it's all kind of out there, it feels like, in one way or another. With images and with the text-to-image use case, the images were never captioned with the intent of allowing you to create images in a text-to-image sort of way. So it's sort of this found dataset concept that, oh, hey, look at all this web-scale data. These captions are super noisy. Obviously we've gotten better at that, filtering, et cetera, over time. But largely it is still kind of derived from this The fact that people happen to, you know, caption their images sometimes. I guess there's maybe just a lot of missing data where like you could say, here's an image and I want, you know, it reminds me of like the bad Photoshop memes online, right? Where people are like, oh, I want my, you know, boyfriend to be over here or whatever. Then the, you know, the person like makes a mockery of their request. But more generally, you know, more like productively, It seems like what's missing is that we just don't have those before and after transformations in image almost at all.

Is there a lack of training data relative to say language where maybe there's vast quantities of structured and good training data? Is that kind of what you're saying?

Yeah, the transformations that you would want to do now that you're starting to imagine what a robust AI tool would look like, are just very hard for humans to do, very rare online, require advanced Photoshop skills historically, and just don't exist and just aren't posted all that much, I would guess.

It turns out that vision is perhaps more prone to being able to succeed with even synthetic data. One benefit of all these images is that it's very easy to make Lots, generate lots of them and annotate them and change things with them. So it turns out that actually you can probably pre-train a good model on ground truth stuff, but then there are all these wonderful other models that you can use to train on probably synthetic data to get a really good new kind of general model that's capable of new things. A really good example of this for folks Is maybe the thing that put me onto this most early was InstructPix2Pix early, I think last year, where so much of it is actually just synthetic data. I think its overall performance was not world-changing or anything, but it gives you kind of a glimpse of what it could be if given— I mean, this was just one researcher at Berkeley, I think under Efros. So I think given a team that really cares, what could it be? How good could it be? So I think that's one thing that might be surprising regarding this debate around training data. The second thing I would say is that part because of the massive revolution with text and in specific some of these text vision models, these multimodal models that are getting built now, whether that's at Google or OpenAI with GPT-4B, or there's some really amazing open source ones. By students in academia. I think this guy named, I don't know, sorry if I butcher the way to pronounce his name. I think his name is Haoshen Lin or something like that. He did LLaMA 1.5 with some other people. And so there's just this, I think going into this year, one thing that I think will probably be true, it's kind of a natural progression from where we are with language is, and you probably need to augment language somehow. Is really great multimodal models. And it just turns out that if you scale up these multimodal models with text understanding and vision, obviously, then you can even do more capable things. It might be surprising how that can help with vision more significantly. Maybe not too surprising. But certainly the language part was very important for these multimodal models to be as powerful as they are. For example, hopefully things like image segmentation get better, but I think captioning gets better. One thing everybody knows in the field for vision, or at least with graphics, is that that LAION dataset is really poorly annotated. It's sort of a best-case effort, but it's very poorly done. I think the folks that made it did the best they could, but the data's not great. And so if you have a multimodal model that's extremely good, the DALL·E 3 paper talked about how that team just completely recaptioned the dataset to be more accurate. And then that significantly improved prompt alignment for the image models. And now you get better prompt alignment. And now suddenly, if you ask the models to— one little test that I like is create an image where you've got 4 bottles and they're numbered 1, 2, 3, 4, but ordered backwards. You try to do these little puzzle challenges for the model. And suddenly these kinds of spatial reasoning tasks become possible. Or if you do recaptioning, text synthesis starts to get better, performance starts to dramatically increase. Now we're starting to see Midjourney and DALL-E 3 be better at text. So it just turns out that actually the overall progression of the entire AI field helps quite significantly with vision. And so I think this is just going to keep happening.

So if I'm understanding correctly how you are expecting things to develop, it's almost kind of a mirror image in some ways to language, where the language progress, that's at least like the kind of canonical breakthroughs, seem to be just scaled up. And then, oh my God, look at this. We've got few-shot learning, these kind of quote unquote emergent capabilities that we didn't train for. And now, of course, we're refining that and we're doing curriculum learning and alignment and a million things. But it was kind of like first the brute force fact that you just dump all the text in, run it, and like amazing, something amazing comes out. And now we'll refine from there. On the vision side, it sounds like you're, the path is more of, okay, let's, we have all these different specialist models and now those are going to help us kind of create the super general dataset, which doesn't exist totally as needed in the wild. And then we can kind of Go Captain Planet with our forces combined. Now we can summon the truly generalist vision model that you're dreaming of.

There are these pros and cons. On one hand, images are just so rich in terms of information relative to language. Just look outside your window for a minute and try to really look at every object. Right now I'm looking at Plants. And like, even just this, even just looking at a plant, you can kind of see like, how do the leaves fall? How do they interact together? How does lighting react to the plant? And some of this is graphics knowledge, but some of this is just world knowledge, right? A plant on a roof or how buildings look or how they're situated, right? So images on one hand are extremely rich in terms of data. They're so rich. But then they lack the right annotation and labeling. And so on the other hand, text perhaps has really great, rich, wonderful labeling inherently. That's just inherent in language anyway. But on the other hand, language is very lossy. We don't have that many words to describe things. And the words certainly are not descriptive enough. So on the other hand, there's kind of cons to this and they have different utilities in our life. I can't say that they're like, it makes sense that this happened, but it just so happens to be that we've got these amazing text models that are starting to lead to amazing multimodal models with vision. And that's probably going to help us fix some of our understanding of the images, like what's really going on in some of these images. Where are things located? Why are they the way they are? And so my guess is like, you know, if you think of vision as, in my mind, I kind of like look at vision as like maybe about a year or 2 behind where we are with language. And it's wonderful actually that these multimodal models are getting so powerful because they're going to be very helpful with vision. They're going to lead to vastly better use cases for vision than they do today.

That's a good guide to what the future might have in store for us. How are you thinking about building a business Through this maturation of the technology? Like, are you trying to grow a lot of revenue today? How much do you care about adoption and your sort of relative position in the market, you know, compared to other options? You know, from a funding standpoint, I know you had some capital already on hand when you pivoted into this, but, you know, I imagine more would be helpful. Like, how are you thinking about What do you need to do to raise more money if you need or want to do that? And just the timing of it, it's like very weird, right? Because we're, it seems like unlike previous technology waves, so much is in the future relative to what you can achieve today. And I wonder how you think about like metrics, milestones, you know, kind of proof points versus this like grand strategy, how you kind of balance those and try to make them work together.

Our overall main quest is to make a unified vision model, whether it's images, video, 3D, et cetera, something that has true general understanding of pixels and such. But I think a very simple thing we're doing this year is we're only going to be working on 2 things. We're going to work on making a really great graphics model for just static 2D images. Because a vast majority of the utilities still gone unsolved. And so, yeah, I mean, I think it's not that complicated. I think in the sense that right now, if you want to do things with Photoshop or Lightroom or Illustrator or whatnot, I'm not an expert at these tools. There was a time where I was doing Photoshop tutorials in high school, trying to compete in logo contests and stuff. Since then, my skill has greatly atrophied. But I'm not an expert at this. And I think a vast majority of humanity is It seems like there's clearly like people that buy things and manipulate and want to buy graphics or manipulate pixels and wish they could and just don't, or just retake the picture to get a better one. And then there's people that have like really amazing skill. They're expert color graders. They know how to get rid of like dangly hairs in your wedding picture. They make logos in Photoshop, whatever. Or they're really amazing illustrators. And it takes a lot of skill to be that good. And so I think to the extent to which we can kind of give more of humanity the ability to do that kind of work, but kind of own, instead of going through a third party, be able to actually do some of these things on their own very easily, I think that will open up a lot of doors for us. And so I think we'll be very focused on just creating images and adding them throughout the year. And I hope that leads to a lot of really happy users for whom I think have no alternative right now other than asking a friend or going somewhere where there's someone that is talented, hiring somebody basically.

That feels like a little bit of a change from last year and maybe from the usual advice of fast iteration, talking to users. This feels a little bit more, and I've heard similar comments from Sam Altman recently where he was like, at YC, I told everyone launch super early, iterate super frequently. And then at OpenAI, we took 4 years to launch anything and we had a massive capital outlay and we didn't really know what the use case was going to be. It sounds like you're kind of shifting a little bit more toward that approach where you're like, build something truly awesome. We kind of know what that is in our gut. And then everything else will kind of follow from there.

We've had, maybe we didn't talk about it last year too much, but kind of had the same plan since the company started. But I do want to say like over the last year, we shipped like 100 things.

Yeah, the product has changed a lot. No doubt about that. Every time I've gone in there, it has been notably more feature rich for sure.

You know, we have a Slack channel where we see every complaint from a user. You know, I don't like the GANs, you change too much. How do you— I get DMs from customers or even friends, hey, how do I make this character consistent? And the combination of these things turns out there's no quick fix to solve these things. I think there are certain class of problems where you accumulate them and you go, well, actually there is no quick fix for this. The worst product that we could make is a product where we have a button that's like the fix hands button. And then there's a Um, fix the eyes button. And then there's like a, you know, um, the character consistency button. Like nobody wants to use a product like that. That was a very complicated product to use. You have to know where everything is and then people are confused and you have to figure out ways to train them.

You got to train a whole language model on top of that.

Right.

And then you know, your language models, like choosing which model to use at what point. And then of course, like now your inference times are going up because you're using each model in the pipeline to fix one thing. And then This model fixes the thing, but then it kind of made some other thing worse. This is not a great experience. And so this is all patchwork stuff that's just not going to lead to a world-class product. And so I think my belief is that we've actually distilled— we know what users are frustrated about every single day with the current state of the art. And we bought a lot of GPUs and we found a lot of really amazing researchers. And so it's not like I think we're going to go solve every imaginable editing and graphics problem day one, or even, or we're going to wait 3 years and we're going to have something. I think that as long as we can make a few things very useful that are generally useful, that attract a lot of users for whom like, oh wow, I didn't know, I'm finally happy someone solved this problem. I think that you can build a great, you can start from something kind of small. That starts to become bigger. And I think a good example of this actually is like if we look at Midjourney and what it looked like kind of early last year compared to what it looks like this year, you know, it might seem surprising that anyone paid for the thing early last year, right? But nonetheless, you know, nonetheless, it was the best at the time. And then over time, you know, these things kind of grow and they sort of mature. And so maybe what we'll have is something that's like a PS2 or a PlayStation 1, but it would still be very fun. play or use rather. And then over time, we'll certainly keep making our version of our models better and better. And so the use cases, I hope, will expand.

One other big picture question I wanted to get your take on. Obviously, we're in this moment of all this stuff is so new. The fallout is just beginning, mostly positive, some negative. And we really don't have new kind of guiding rules or principles in many cases even for How should we handle all of this? What should the rules be? I just did another episode on New York Times versus OpenAI, and I guess my super big picture question to you is, do you have any thoughts at this point as to what the rules should be? That could be at the training data level. I think Japan is kind of going this direction where they're going to make everything available for everybody to train on. You could imagine people should be compensated if they're included. You could imagine people should be able to opt out, but not necessarily be entitled to compensation. You could imagine like, I should not be able to ask for everything in the style of Greg Rutkowski, or maybe that's okay. You know, can I make Mario and Luigi on Playground AI? There's just so many questions. I don't expect you to answer them all, but I wonder what your kind of emerging sense for what the rules of the road for the generative AI era should be.

I try to sometimes put myself in the shoes of, uh, you know, let's say the artists or the people making these images. photographers, whoever, because obviously we have a rational self-interest on training on data like that. We were the first site. I read Greg Rutkowski's story, his kind of op-ed, and tried to understand him better to the extent I can. I've not talked to him, but I looked at his page on DeviantArt. I get his style. And I think there are 2 things that are kind of happening. One is Nobody, people don't really want to make art like Greg Rutkowski. It's not very exciting to make. The users that use these products are actually people that think of themselves as artists in some ways. And nobody, people that are artists don't really want to copy another artist, claim their art. There's no pride in that. It's not like these people are making images and they're really selling them for thousands of dollars or anything. So they're not taking great— I don't think a vast majority of are taking pride in that. And the other thing is, you know, after I read his article, I wanted— we were the first site ever, and I think the only site still to this day, for whatever reason, where if there was a prompt on our site and someone references his name, we directly link back to his page. We actually give him— we actually call it additional credit. Literally say, we say additional credit, say Gregor Kousoumou, we link back because one of the things he said was that in Google search, Now it's like people can't even find him. And I thought, well, maybe one way that we could at least help him is helping people find him, find his actual art, pay for it, buy it, whatever. At least give him fame, some other way to evolve in a world where like maybe if the pain was growing for him, maybe we could ease that a little bit. So I think there's small things that we can do. I'm not saying that that like fixes and alleviates the issue for Greg in any shape or form, just saying that I think there might be some small things that we can do. I think some bigger things, you know, one, the other reality is that, is that, you know, it's not like you can't make Mario and Luigi in Photoshop or Illustrator, right? And what, and the rules around that are like, of course you and I can gain the skills and we can make Mario, Luigi, or Mickey Mouse or whatever, but we can't just go run around like starting to sell t-shirts with like Nintendo branding either, right? And so I think there's like another world where there's kind of a balance point where like, It might be generally okay to make things, right? In fact, many brands are perfectly fine with fan art or whatever. But when then there's kind of this question of commercial use, that's where things kind of like stop. And I think that's like a fair area to say, oh, you know, people are just kind of having fun. But, you know, when they start selling t-shirts in the thousands with kind of like a Bizarro World looking Mickey Mouse, maybe that's not okay. Right. And so I think that that's probably okay too. I think it's going to be very hard, even if the models are not trained necessarily on Mickey Mouse or Mario and Luigi, I think it's going to be very hard to get these models— at some point, the models will have some representation of it, even if it hasn't necessarily strictly seen— even if you try your best to comb through the dataset, there'll always be something missing, or it'll just have some way of understanding Concepts of it. So I think that's a very impractical thing. I think the genie is a little bit out of the bottle, but I think it might be better to actually talk about use as opposed to strictly training data. That's my view. I also think the last part of this is maybe how fast it goes. If this whole thing is very gradual, then I think probably society will find some way to assimilate to it. I think if If it's vastly faster than that, if it's like a step function thing each time, then I think that we definitely need to do something about that. I think that who it's impacting very much needs to be considered. It cannot just be thrown away and we can't ask people to evolve and upskill something that they've trained on doing their whole— some craft that they've been doing for a decade or 2 and ask them to upskill and evolve and catch up to the times within a couple of years. So I don't think that's a real rational path. So that's kind of like my rough triangulation of things. My feeling around like rev share on training data, it just seems like it's going to disappoint people more than it's going to, you know, be practical and helpful. So I'm a little less sure on how we would really achieve that.

Yeah, it's tough, right? If you are the artist that used to charge however many, or even just, you know, the PowerPoint technician that used to charge however many dollars for a job, And now that can be done for a cent or whatever on a technology tool. Like the rev share on that cent is not, it's not going to buy too much.

Yeah, this, I make music. So I sometimes think about like, what if people would train on my music? What if I was a bigger artist and they trained on music? Kind of how would I feel? And like this sort of rev share model, like streaming revenues are really bad for music artists. This like penny per thing always makes the artist feel like they're greatly undervalued. It leaves tons of resentment in the industry. So, you know, I'm not so sure about that. On the other hand though, I think artists very much crave like an audience and like ways that they can— like making Greg Rutkowski's art is definitely not the same as like buying his art. I think that's like a very different relationship people have with artists. If I make a song that's exactly like Avicii, people will be like, you just sound like Avicii. But it doesn't mean my song will become big in the world because people have built the relationship with the artist. And so I think that something that's really catering to helping give a spotlight to the artists, the real people involved in doing some of this stuff, I think would be very helpful because I think there's still a big audience of people that want that. Even if you could reproduce the same exact image or something like it.

So how fast do you think it's gonna go? And do you think we are headed for kind of a new social contract? I mean, I'm getting like UBI vibes a little bit from a couple of your comments there.

No, I don't, I don't have any strong opinions around how, how to solve it. I, I think that there are much better minds, much better people are well-versed in trying to understand like what happens with technological change. How do we Retool people? How do we, you know, maybe it becomes like a tax.

I don't know.

Yeah, maybe it is kind of like a UBI tax sort of thing for certain industries. Certainly investing in helping people like upskill. You know, one funny thing out of the art debate was I once DM'd with someone who really hated the AI art stuff, I think like a year and a half ago. And some of these people don't want to retool. They just like enjoy. One of them said, I just enjoy drawing. I don't want to retool. I don't want to get new skills. I just enjoy this craft. And so I think there's also a practical reality about that. You can't ask people to retool when they don't enjoy the craft anymore because they enjoyed it the way that it was. So I definitely think there'll be some version of this. I don't know how you distribute money like that. I don't have any sense. I do think that the pace is varied. It's going to be varied. very fast. So I don't think it can be ignored. I think it would be— I think we'll be making a very big mistake as a technology industry if we just sort of say, throw caution to the wind and kind of go, we're just going to do this very quickly. We don't care who it impacts. I think that the last decade of technology and the tech industry in particular has showed that when we do that, we really disenfranchise people. And that creates Kind of like a bad circumstance for our industry. And I hope that we've learned that lesson.

Yeah, I totally agree. Okay. One more little follow-up because I can't resist. I've been thinking lately, not so much for the art side, but there's probably a lot of commonalities. What are the minimum standards that AI application developers should be expected To uphold, meaning, you know, if you didn't do this, you are worthy of like shame within the industry, even if there are no, you know, laws yet on the books to control what happens. I wonder if you have any thoughts as to, you know, kind of what AI developers should demand of one another. Presumably there will be actual rules coming, but as we kind of ease into that and hopefully maybe even can inform what they might look like so they don't end up being dumb, which is a very real risk, I think.

Yeah.

What would be a good sort of self-governing, maybe just the kernel of a self-governing standard that we can all kind of say, hey, this is, we got to do this, guys, at least?

There's already kind of like a pretty reasonable consensus around safety in the world. Unfortunately, safety kind of got wrapped around overloaded with like alignment and stuff. But, and so I try to come up with like a different word sometimes for situations. Anyone operating these models has seen extremely bad actors. We all have our version of a story that just seems extremely, extremely bad. And so there's just general evil out there and people, how they're using these tools. And so one of the things that we did at Playground is the existing state-of-the-art safety filter that stops, regardless of your point of view, we don't personally want nude content on our site, for example. and other forms of content like that. We ended up training a new state-of-the-art safety filter that went further. Of course, it helps us. We don't have to moderate as much, but also it just reduces significant evil that tends to occur in the world. Not that nude images in particular are evil, but there are definitely classes of images that are more evil or illegal in our country than other classes of images. And so I think they're You know, I just think like people using images for like deepfakes and using it out of revenge and how we're going into like a new election year, how things are going to be manipulated, how we're going to explain to our aunt or our uncle or our parents or a brother and sister that like that thing that they saw on the internet was actually not real, even though maybe it fit their narrative, it actually technically wasn't real. So I think just combating just the sheer disinformation, misuse, evil that might occur using these tools, because the tools tend to be extremely powerful. And so they have incredible amounts of value for the world, but it turns out that that can totally be used for nefarious things. And so I just think that that seems like the most basic thing that we can all do. And I kind of wish, one thing that I wish is that The safety parts of this business were probably— I wish they were a lot more open, actually. OpenAI has a moderation model. We use it. It's free. We use it. They don't charge you anything for it unless you use too much of it. But too much of it is a very high threshold. And I just wish that maybe we'd have a far more open collaboration around any of this kind of stuff because we all have different— kinds of data and we would all do better if we worked a little bit better together on this. And it would be better to have a state-of-the-art model. It's not that difficult for nefarious actors to test these things and try to work around them. I think we could beat this cat and mouse race if there was a larger set of companies working on an open set of safety models. I don't think it would take away from anyone's core IP or anything like that. I think the world would be better off for it. And that one might be a very— the model ends up— the model's performance would end up being kind of like a definitive answer to your question around what should we all be doing that is like a minimum moral standard. Well, actually the minimum moral standard is high dimensionality too. It could be encapsulated in this model that we all work on together and we all contribute together and the best researchers in the field work on together.

It's a beautiful vision. The opportunity and the peril of generative AI, I think very well articulated there. I think that's a great note to end on. I will say again, Suhail Doshi, thank you for being part of the Cognitive Revolution.

Thanks for having me.

It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at tcr@turpentine.co, or you can DM me on the social media platform of your choice.