Who Will Govern the Future of AGI? w/ Emad Mostaque (Stability AI Founder) | X Spaces

Wouldn't it be great to manage your portfolio on one platform? Well, now you can trade all in one place on Robinhood. That means you can trade individual stocks and ETFs and also buy and sell crypto using seriously powerful and intuitive tools at one of the lowest costs on average without needing to manage multiple apps. Robinhood makes withdrawing and depositing crypto seamless. Send crypto to your Robinhood account or send crypto from your Robinhood account to other wallets without deposit or withdrawal fees from Robinhood. Trade all in one place. Get started now on Robinhood Trading. Crypto involves significant risk. Crypto trading is offered through an account with Robinhood Crypto llc. Robinhood Crypto is licensed to engage in virtual currency business activity by the New York State Department of Financial Services. Crypto held through Robinhood crypto is not FDIC insured or SIPC Protected network fees may apply to crypto transfers. Crypto transfers may not be available to all customers. Investing involves risk including loss of principal securities trading is offered through an account with Robinhood Financial LLC member sipc, a registered broker dealer.

You're about to make a trade. Which u do you listen to?

Is it get optioning those options or.

Let'S do a little research. Learn more@finra.org TradeSmart.

Hey Maud, good to hear you.

It was always a pleasure.

Yeah.

So where are you today?

I'm in London, other side of the planet. I'm in Santa Monica. It's, it's been quite the extraordinary game of ping pong out there these last four or five days.

I didn't think the first thing that AI would disrupt would be reality tv. Right?

Yeah, yeah it's been, it's been fascinating how, how, how X has become sort of the go to place to find out the latest of where Sam is working and what's going on with the AI industry.

So yeah, you found the notifications and the way it goes. I mean this is the thing like what else will move at the speed of this? Like I was saying someone recently, AI research doesn't really move at the speed of conferences or even PDFs anymore. Right. You just wake up and you're like oh, it's 10 times faster. So I think that's why X is quite good. I actually like unfollow just about everyone and so the AI algorithm find the most interesting things for me. So like 10 people I follow and it's actually working really well. It's getting better. Well.

It has been. I've enjoying the conversation. It really feels like you're inside a intimate conversation among friends as this is going back and forth, I think this entire four or five days has been an extraordinary up close intimate conversation around governance and around what's the future of AI because honestly as it gets faster and more powerful, the cost of missteps is going to increase exponentially. Let's begin here. You've been making the argument about open source as one of the most critical elements of governance for a while now. Can you just. Let's hop into that.

Yeah, I think that you know, open source, the difficult one because it means a few different things like is it models you can download and use? Do you make all the data available and free? And then when you actually look at what all these big companies do, all their stuff is built on open source basis, you know, it's built on the transformer paper. It's built on like the new model by kai Fu Lee and 01 AI is basically llama like it's actually got the same variable names and other things like that plus a gigantic supercomputer. Right.

The whole conversation has been, you know, how important is openness and transparency and what are the governance models that are going to allow the most powerful technology on the planet to enable the most benefit for humanity and the safety. So I mean you've been thinking about this and speaking to transparency, openness, governance for a while and could you, I mean what do you think is going to be, what do you think is we need to be focused on? Where do we need to evolve to?

It's a complicated topic. I think that most of the infrastructure of the Internet is open source, Linux, everything like that. I think these models, it's unlikely that our governments will be run on GPT 7 or Bard or anything like that. How are you going to have black boxes that run these things? I think a lot of the governance debate has been hijacked by the AI safety debate where people are talking about AGI killing us all and then there's this precautionary principle that kicks in. It's too dangerous to let out because what if China gets it? What if someone builds an AGI that kills us all? It'd be great to have this amazing board that could pull the off switch, you know. Whereas in reality I think that you're seeing a real social impact from this technology and it's about who advances forward and who's left behind if we're thinking about risk. Because governance is always about finding, as you said, the best outcomes and also mitigating against the harms. Right. And there's some very real, amazingly positive outcomes that are now emerging that people can agree on, but also some very real social impacts that we have to mitigate things.

So, I mean, let's begin. How is stability governed?

Stability is basically governed by me. So I looked in foundations and daos and everything like that, and I thought to take it to where we are now, it needed to have very singular governance. But now we're looking at other alternatives.

And what do you think it's going to, you know, where would you head in the future? I mean, let's, let's actually jump away from this. In particular, what do you recommend? The most powerful technologies on the planet, how should they be governed? How should they be owned? You know, where. Where should we be in five years?

I think they need to be public goods that are collectively owned and then individually owned as well. So, for example, there was the tweet kind of storm, the kind of I am Spartacus or his name is Robert Paulson in the OpenAI team saying, OpenAI is nothing without its people. Stability. We have amazing people. 165 top researchers without its people. We're open models used by hundreds of millions. It continues. And if you think about where you need to go, you can never have a choke point on this technology. I think it becomes part of your life. Like the phrase I have is not your models, not your mind. So these models, again, are just such interesting things. Take billions of images or trillions of words and you get this file out. That can do magic, right? Trained on magic sand. I think that you will have pilots that gather our global knowledge on various modalities, and you'll have co pilots that you individually own that guide you through life. And I can't see how that can be controlled by any one organization.

You've been on record talking about having models owned by the citizens of nations. Can you speak to that a little bit?

Sure. So we just released some of the top Japanese models, from visual language to language to Japanese SDXL as an example. So we're training for half a dozen different nations models now. And the plan is to figure out a way to give ownership of these data sets and models back to the people of that nation. So you get the smartest people in Mexico to run a stability Mexico or maybe a different structure that then makes decisions for Mexicans with the Mexicans about the data and what goes in it. Because everyone's been focusing on the outputs. The inputs actually are the things that matter the most. The best way I've thought about Thinking of these models is like very enthusiastic. So hallucinations is I'm just trying too hard. A lot of the things about like, oh, what about these bad things the models can output, it's about what you input and so what you put into that Mexican data set or the Chinese or Vietnamese one will impact the outputs. And there's a great paper in Nature Human Behavior Today about that, about how foundational models are cultural technologies. So again, how can you outsource your culture and your brains to think other countries to people that are from a different place? I think it eventually has to be localized.

Yeah, I think one of the points you said originally is we have to separate the issue of governance versus safety and alignment. Are they actually different?

So I think that a lot of the safety discussion or this AGI risk discussion is because the future is so uncertain because it is so powerful, right? And we didn't have a good view of where we're going. So when you go on a journey and you don't know where you're going, you'll minimize for maximum regret, you'll have the precautionary principle. And then that means you basically go towards authority, you go towards trying to control this technology when it's so difficult to control and you end up not doing much, you know, because everything could go wrong. When you have an idea of where we're going, like you should have all the cancer knowledge in the world at your fingertips, or climate knowledge or anybody should be able to create whole worlds and share them. Then you align your safety discussions against the goal, against the location that you're going to. Again, just like setting out on a journey. And I think that's a big change. Similarly, most of the safety discussion has been on outputs, not inputs. If you have a high quality data set without knowledge about anthrax, your language model is unlikely 2 tell you how to build anthrax, you know, So I think that's it. And transparency around that will be very useful.

So that's a, you know, let's dive into that safety alignment issue for a moment because it's an area you and I have been talking a lot about. So Mustafa wrote a book, Mustafa Selimanders wrote a book called the Coming Coming Wave in which he talks about containment as the mechanism by which we're going to be making sure we have safe AI. You and I have had the conversation of. It's really how you educate and raise and train your AI systems in, in giving making sure that there's full transparency and openness on the data Sets that are utilized. Do you think containment is an option for, for safety?

No, not at all. Mike, a number of leaders say what if China gets open source AI? The reality is that China, Russia, everyone already has the weights for GPT4 because they just downloaded it on a USB stick. You know, you would know that there's been compromised, right? There's no way they couldn't. The rewards are too great. And there is a absolutely false dichotomy here and thing a lot of the companies want you to believe that giant models are the main thing and you need to have these gigantic ridiculous supercomputers that only they can run. I mean, look, we run gigantic supercomputers, but the reality is this, the supercomputers and the giant trillion zillion data sets are just a shortcut for bad quality data. It's like using a hot pot or sous vided ing a steak that's bad quality. You cook it for longer and it organizes the information with stable diffusion. We did a study and we showed that basically 92% of the data isn't used 99% of the time. Because now you're seeing this with for example, Microsoft's Phi release, it's trained entirely on synthetic data. Dall E3 is trained on RVAE and entirely synthetic data. You are what you eat. And again, we cooked it for longer to get past that. But the implications of this are that I believe within 12 to 18 months you'll see GPT4 level performance on a smartphone. How do you contain that? And how do you contain it when China can do distributed training at scale and release open source models? So Google recently did 53,000 TPU training run on their V5Es. The new V5Es, their TPUs are very low powered relative to what we've seen. But again, you can do distributed dynamic training. Similarly, like we funded Hivemind and we've seen Google DeepMind just did a new paper on localization through distributed training. The models are good, fast enough and cheap enough that you can swarm them and you don't need giant supercomputers anymore. And that has a lot of implications. And how are you going to contain that?

So coming back to the question of do you mandate training sets? Do you, does you know, does the government set out what all companies should be utilizing and mandate? If you're going to have a aligned AI, it has to be trained on these sets. How do we possibly govern that?

Look, we have food standards, right, for ingredients. Why don't you have data standards for the ingredients that make up A model. It's just data compute and some algorithms. Right. And so you should say here are the standards and then you can make it compulsory. That will take a while. Or you can just have an ISO type standard. This is good quality model training, good quality data, you know, and people will naturally gravitate towards that and it becomes the default.

Are you working?

I think.

Are you working towards that right now?

Yeah. I mean, look, we spun out Eleutherai as an independent 501C3 so they could look at data standards and things like that. Independent of us. Kind of the opposite of OpenAI. This is something I've been talking to many people about and we're getting national data sets and more so that hopefully we can implement good standards similar to how we offered opt out and had a billion images opted out of our image dataset because everyone was just training on everything. Is it required? No. But is it good? Yes. And everyone will benefit from better quality data. So there's no reason that for these very large model training runs, the data set should not be transparent and logged. Again, we want to know what goes into that. Again, if we have the graduate analogy, what was the curriculum that the graduate was taught at? Which university did they go to? It's something that we'd want to know. But then why do we talk to GPT4 when we don't know where it went to university or where it's been trained on? Bit weird that, isn't it?

What do you think the lesson is going to be from the last four days?

I'm just confused. I don't know who was against who or what. I think I just posted are we against misalignment or Moloch or anything like that? I think probably the biggest lesson is it's very hard to align humans. Right. And the stakes are very large. Like why is this so interesting to us? Because the stakes are so high.

You tweeted something that was serious and unfortunately funny, which was how things we align AI with human humanity's best interests. If we can't align our company's board with its employees best interest.

Well, the thing is it's not the employees best interest. It's like the board was set up as a lever to ensure the charter of OpenAI. So if you look at the original founding document of OpenAI from 2015, it is a beautiful document talking about open collaboration, everything. And then it kind of changed in 2019. But the charter still emphasizes cooperation, safety and fundamental. Posted about this back in March when I said the board and the government structure of OpenAI is weird. Like, what is it for? What are they trying to do? Because if you say you're building AGI in their own road to AGI, they say this will end democracy, most likely.

I remember reading that.

Because democracy, yeah, there's no way democracy survives AGI because either obviously it'll be better and you get it to do it, or it can persuade everyone or we all die or it's utopia forever. Right?

Abundance, baby.

Yeah, there's no yeah, well, this thing. But anyway, regardless, there's no way it survives AGI. There's no way capitalism survives AGI. The AGI will be the best trader in the world, right? And it's like, who should be making the decisions on the AGI, right? Assuming that they achieves those things. And that's in their own words. So I think that people are kind of waking up to, oh, there's no real way to do this properly. And previously we were scared of open and being transparent, everyone getting this, which came with the original thing of OpenAI. And now we're scared of who are these clowns? You know, put it in the nicest way because this was ridiculous. Like, you see better politics in a teenage sorority, right? And it's fundamentally scary that unelected people, no matter how great they are, and I think some of the award members are great, should have a say in something that could literally upend our entire society according to their own words. I find that inherently anti democratic and illiberal.

At the end of the day.

You.

Know, capitalism has worked and it's the best system that we have thus far. And it's a self, you know, it's, it's built on self interest and built on continuous optimization and maximization. I'm still wondering where you go in terms of governing these companies at one level, internal governance, and then governing the companies at a national and global level. Has anybody put forward a plan that you think is worth highlighting here?

Not really. I mean, organizations are weird. Artificial intelligence, right? They have the status of people and they're slow, dumb AI and they eat our hopes and dreams. That's what they feed on, I think. But this AI can upgrade them, it can make them smarter. They can. How do you coordinate? And from a mechanism, design, perspective, it's super interesting. Like in markets, I think we will have AI market makers that can tell stories. Like the story of Silicon Valley bank went around the world in two seconds. The story of OpenAI goes around. AI can tell better stories than humans. It's inevitable. I think that gives hope for coordination, but then also it's dangers of disruption.

I want to double click one second on the two words that you use most, openness and transparency, and understand fully what those mean. One moment. Because you know, and the question is not only what they mean, but how fundamental need to be. So openness right now in your definition in terms of AI means what.

It means different things. But different things. Unfortunately, I don't think it means open source. I think for me, open means more about access and ownership of the models so that you don't have lockstep, like you can hire your own graduates as opposed to really relying on consultants. Transparency comes down to, I think for language models in particular. I don't think this holds for media models. You really need to know what it's been taught. That's the only way to safety. Like you should not engage with something or use something if you don't know what its credentials are and how it's been taught. Because I think that's inherently dangerous as these get more and more capabilities. And again, I don't know if we get to AGI. Like if we do, I think it'll probably be like Scarlett Johansson and her, you know, like just to give by and thanks for gps. But assuming you don't, you still need transparency.

So not again.

How can any government or regulated industry not run on a transparent model? They can't run in black boxes.

I get that and I understand the rationale for it. But now the question is, can you prove transparency?

I think that again, a model is only three things, really. It's the data, the algorithm and the compute. And then they come and the binary file pops out. Then you can tune it with RLHF or DPO or genetic algorithms or whatever. But that's really the recipe, right? And so the algorithms, you don't need algorithmic transparency here versus classical AI because they're very simple. One of our fellows recreated the palm 540 billion parameter model. This is Lucid Rains on GitHub. You look at that if you're a developer and you want to cry, it's GitHub. It's crazy. In 206 lines of PyTorch. And that's it. The algorithms are not very complicated. Running a gigantic supercomputer is complicated. And this is why they freaked out when Greg Brockman kind of stepped down, because he's one of the most talented engineers of our time. Built this amazing gigantic clusters. And then the data and how you structure data is complicated. So I think you can have transparency there because if the data is transparent, then who cares about the supercomputer? Who really cares about the algorithm?

Now let's talk about the next term, alignment here. Alignment's thrown around in lots of different ways. How do you define alignment?

I define alignment in terms of objective function. So YouTube was used by the extremists to serve ads for their nastiness. Right. Why? Because the algorithm optimized for engagement, which then optimized for extreme content, which then optimized for the extremists. Did YouTube mean that? No, but they were just trying to serve ads up. Right, but it meant it wasn't aligned with its users interests. And so for me, if you have these technologies that we're going to outsource more of our mind, our culture, our children's futures to that are very persuasive, we have to ensure they're aligned with our individual community and societal best interests. I think this is where the tension with corporations will come in. Because whoever licenses Scarlett Johansson's voice will sell a lot of ads. You know, they can be very, very persuasive. But then what are the controls on that? No one talks about that. The bigger question of alignment is not killerism, making sure the AI doesn't kill us. But again, I feel that if we build AI that is transparent, that we can test that people can build mitigations around, we are more likely to survive and thrive. And also I think there's a final element to here which is whose alignment? Yes, different cultures are different, different people are different. What we found with stable diffusion is that when we merge together the models that different people around the world have built, the model gets so much better. I think that makes sense because a monoculture will always be less fragile than the diversity. Again, I'm not talking about in the DEI kind of way. I'm talking about it in the actual logical way. So we have a paper from our reinforcement learning lab called carpa called qdhf qdaif Quality and Diversity through Artificial Intelligence Feedback. Because you find these models do get better with high quality and diverse inputs, just like you will get better if you have high quality and diverse experiences. And that's something that's important that will get lost if all these models are centralized.

You and I have had a lot of conversations about timelines here. We can get into a conversation of when and if we see AGI, but we're seeing more and more powerful capabilities coming online right now that are going to cause a lot of amazing progress and disruption. How much time do we have? Imad and we had a conversation when we were together at FII about.

The.

Disenfranchised youth coming off Covid. So let's talk one second about timelines. How long do we have to get our shit together, both as AI companies and investors and governors of society? The speed here is awesome and frightening. How long do we have.

Everything everywhere, all at once, Right? We don't have long like AGI timelines for ever. Definition of AGI, I have no idea. It will never be less than 12 months. Right? Because it's such a step change. Let's put that to the side. Okay. Right now, everyone that's listening, are you all going to hire the same amount of graduates that you hired before? The answer is no. Some people might not hire any because this is a productivity enhancer and we have the data for that across any type of knowledge industry. You just had a great app that you can sketch and it does a whole iPhone app for you. Right. I got on record and saying there are no programmers. We know in five years. Why? Why would there be? What are interfaces? You had a 50% drop. I just post that on my Twitter in hiring from Indian IITs. That's crazy. So what you're going to have in a couple of years is around the world at the same time, these kids that have gone through the trauma of COVID highly educated STEM programming, accountancy law, simultaneously, people will hire massively less of them because productivity enhances and you don't need as many of them. Why would you need as many paralegals? And that for me is a gigantic societal issue. And the only thing I can think of is to stoke open innovation and the generative jobs of the future through open source technology. Because I don't know how else we're going to mitigate that. Because, you know, Peter, you're a student of history. What happens when you have large amounts of intelligent disenfranchised youth?

We've had that happen a few times. We just had Arab Spring not long ago. Revolt. Yeah, yeah. Civil war. If not inter.

You know, international law is a good way to soak up the excess youth. Yep. But it's not pleasant. So not pleasant for society. And fundamentally, the cost of information gathering organization has collapsed. Like you look at stable video that we released yesterday. Right. It's going to get so much better so quickly. Just like stable diffusion, the cost of creating movies decreases, the demand for quality stuff increases. But there's a few years where demand and supply don't match. And that's such a turbulent thing to navigate. That's one of the reasons I'm creating stabilities for different countries so the best and brightest from each can help navigate.

I loved your.

And people don't talk about.

I loved your idea that the, that the stability models and systems will be owned by the nation. In fact, one idea that I heard you say, which I thought was fantastic was you graduate college in India, you're an owner in that, in that system you graduate from in Nigeria, you're an owner in that system basically to incentivize people to complete their education and to have them have ownership in what is ultimately the most important asset that nation has. And talk about it as infrastructure as well. I think that's an important analogy that people don't get.

This is the knowledge infrastructure of the future is the biggest leap forward we have because you'll always have a co pilot that knows everything in a few years and that can create anything in any type. But it must be embedded to your cultural values and you can't let anyone else own that. So it is the infrastructure of the mind and who would outsource the infrastructure to someone else? So that's why I think Nigerians should own the models of Nigerians for Nigerians and it should be the next generation that does that. You know, that's why you give the equity to the graduates. That's why you list it, that's why you make national champions. Because again, that has to be that way. This is far more important than 5G. And this gives you an idea of the scale. We're just at the start, the early adopter phase. A trillion dollars was spent on 5G. This is clearly more important. More than a trillion dollars will be spent on this. And again, it flips the world. And so there is huge threat for our societal balance. And again, I think open is a potential antidote to create the jobs of the future. And there's huge opportunity on the side because no one will ever be alone. And we can use this to coordinate our systems, give everyone all the knowledge that they need at their fingertips and help guide everyone if we build this infrastructure correctly. And again, I don't see that how that can be closed.

AGI, you know, the conversation and the definition of AGI has basically been all over the place. Ray Kurzweil's prediction has been for 30 years that it's 2029. Again, that's a blurry line of what we're trying to, you know, target. But Elon's talked about anywhere from 2025 to 2028. What are you Thinking, how do you, what's your, what's your timeline for, you know, even digital superintelligence?

I honestly have no idea, like people looking at the scaling laws and applying it. But as I've said, data is the key. And it's clear that we already have. Like you could build a board GPT and it'd be better than most corporate boards. Right. So I think we're already seeing improvements over the existing. One of the complications here is swarm AI. So even like is the whole thing like a duck sized human or 100 human sized ducks? Right. We're just at the start of swarm intelligence and that reflects and represents how companies are organized. Andre Karpath, he has some great analogies on this in terms of the new knowledge os and that could take off at any time. But the function and format of that may not be this whole western anti compromised consciousness that we think of, but just incredibly efficient systems that displace existing human decision making. Right. And so there's an entire actual range of different AGI outcomes depending on your definition. And I just don't know. But I feel again like I wake up and I'm like, oh look, it's sped up 10 times the model, you know, like I'm just not. No one can predict this.

But there is a point at which, I mean, we're heading towards an AI singularity. You know, using the definition of a singularity as a point after which you cannot predict what's coming next. And that isn't far away. I mean, how far out is it for you? A year, two years?

I think you're heading forward in the next few years. But like I said, every company, organization, individual has an objective function. My objective function is to allow the next generation to navigate what's coming in the optimal way and achieve their potential. So I don't want to build an AGI. I don't want to do any of this. Amplified human intelligence is my preference. And trying to mitigate against some of the harms of these agentic things through data transparency, good standards and making it so people don't need to build gigantic models on crap, which I think is a major danger if even if not from aego. But again, we just don't understand because it's difficult for us to comprehend superhuman capabilities. But again, we're already seeing that in narrow fields. We already know that it's a better rider than us. We already know that it can make better pictures than us and a better.

Physician and a better educator and a better surgeon and a better everything.

Yeah, and again, I think it's this mythos of these big labs being AGI focused, whereas you can be better than us in like 5% of the stuff that humans can do. And that's still a massive impact on the world. And it can still take over companies and things like that. Right? Like if you take over a company then you can impact the world. And there's clearly with a GPT four or a thousand of them orchestrated correctly that can call up people and stuff, you wouldn't know, it's not the CEO. You know, I can make an migpt and then I won't have to make all these tough decisions and any that and most of my decisions aren't that good. So it'd probably be better. So I think that we're getting to that point. It's very difficult and the design patterns are going fast. We're at the iPhone 2G, 3G stage, got copy and paste and we just got the first stage as well of this technology, which is the creation step. It creates stuff. The next step is control and then composition, where they're annoyed because ChatGPT doesn't remember all the stuff that you've written that won't be the case in a year. And the final bit is collaboration, where these AIs collaborate together and with humans to build the information superstructures of the future. And I don't feel that's more than a few years away and it's completely unpredictable what that will create.

Let's talk about responsibility that AI companies have for making sure that their technology is used in a pro human and not a disruptive fashion. Do you think that is a responsibility of a company, of a company's board, of a company's leadership? How do you think about that?

Again, with the corporatist capitalist system, it typically isn't because you're maximizing shareholder value and there aren't laws and regulations. Which is why I think there's a moral, a social and legal regulatory aspect to this. Companies will just look at the legal regulatory, in some cases they'll just ignore them. Right. But I do think again, we have a bigger moral and social obligation to this. This is why I don't subscribe to EA or EAC or any of these things. I think it's complicated and it's hard given the uncertainty. And how does technology proliferate? And you've got to do your best and be as straight as possible to people about doing your best because none of us are qualified to understand or do this and none of us should be trusted to have the power over this technology, right? You should be questioned, you should be challenged with that. And again, if you're not transparent, how are you going to challenge?

When I think of the most linear organizations on the planet, I think of governments, maybe religions, but governments, let's leave it there. How can, you know, let's talk about Western government. At least the us, I would have said Europe, but I'll say the UK and Europe, what should they be? What steps should they be taking right now? You know, if you were given the reins to say, how would you regulate? How would you. What would you want them to do or not do?

I believe it's a complicated one. So I signed the first FLI letter. I think I was the only AI CEO to do that back before it was cool because I said, I don't think AGI will kill us all, but I just don't know. I think it's a conversation that deserves to be had and it's a good way to have that conversation. And then we flipped the wrong way where we went overly AI, death risk and other things like that. And governments were doing that at the AI Safety Summit in the uk and then we had the King of England come out and say, this is the biggest thing since fire. I was like, okay, that's a big change, right? The King of England said it, so I must be on the right track. But I think if you look at it, regulation doesn't move fast enough. Even the executive order will take a long time. The eu, things will kind of come in. Instead, I think that governments have to focus on the tangibles, AI killerism. Again, it can be addressed by considering this is infrastructure and what infrastructure do we need to give our people to survive and thrive? The US is in a good initial place with the CHIPS act, but I think you need national data sets, you need to provide open models to stoke innovation and think about what the jobs of the future are. Because things are never the same again. You don't need all those programmers when Copilot is so good. And you're moving Copilot to the level above, which is compositional Copilot and then collaborative copilot, right? You'll be able to talk and computers can talk to computers better than humans can talk to computers. So we need to articulate the future on that side, but then the other side. One of the examples I give is a loved one had a recent misdiagnosis of pancreatic cancer. You know, I talked about this and the loss of agency you feel. And many of you on this call, will have had that diagnosis. The near and dear is huge. And then I had a thousand AI agents finding out every piece of information about pancreatic cancer. And then after that I felt a bit more control. Why don't we have a global cancer model that gives you all the knowledge about cancer and helps you talk to your kids and connects with other people like you, not for diagnosis or research, but for humans. This is The Google Med Palm 2 model, for example, that outperforms humans in diagnosis, but also empathy. And what if we armed our graduates to go out and give support to the humans that have been diagnosed in this way? That makes society better and it's valuable. And that's an example of a job of the future, I think. I don't believe in ubi. I think even universal basic jobs as well.

Universal basic opportunity, right?

Universal basic opportunity, you know, space jobs. But then policymakers need to think about it now because the graduate unemployment wave is literally a few years away. And it will happen.

Yeah, that is, I mean, when I think about what I parse the challenges we're going to be facing in society into a few different elements. I think, you know, what we have today is amazing. And if generative AI froze here, we'd have an incredible set of tools to help humanity across all of its areas. And then we've got what's coming in the next, you know, zero to five years. We've talked about patient zero perhaps being the US elections and the, you know, I think you had said, you know, it was Cambridge Analytica that required interference. Now it's any kid in the garage that could play with the elections, that's a challenging period of time. And this graduate unemployment wave, as you mentioned, coming right on its heels, that's, you know, the question becomes, is the only thing that can create alignment and help us overcome this AGI at the highest level, meaning it is causing challenges, but ultimately is a tool that will allow us to enable to solve these challenges as well.

I mean, that's a crazy thought, right? Like all this stuff is crazy. Like the sheer scale and impact of it. And you know, these discussions, we had them last year, Peter, and now everyone's like, yeah, that makes sense. Like, oh, wow, right? It may be AGI, maybe these coordinating automated story makers and balances for the market, right? Next year there's 56 elections with 4 billion people heading to the polls.

What could possibly go wrong?

What could possibly go wrong?

You know, oh, my God.

But again, the technology isn't going to stop. Like, even if stability Puts down things. If OpenAI puts down things, it will continue from around the world because you don't need much to train these models. Again, the supercomputer thing is a myth. You've got another year or two where you need them. You don't need them after that. And that is insane to think about.

You just released Stability video. Congratulations. Our stable video diffusion and I'm enjoying some of the clips. How far are we away from me telling a story to my kids and saying let's make that into a movie?

Almost two years. Two years away. So this is a building block is the first creation step. And then like I said, you have the control step, composition and then collaboration and self learning systems around that. So we have ComfyUI, which is our node based system where you have all the logic that makes up an image. Like you can take a dress and the pose and the face and it combines them all and it's all encoded in the image because you can move beyond files to intelligent workflows that you can collaborate with. If I send you that image file and you put it into your comfy ui, it gives you all the logic that made that up. How insane is that? Right, so we're going to step up there and what's happened now is that people are looking at this AI like instant versus again the huge amount of effort it took to take this information, structure it. But the value is actually in stuff that takes a bit longer. Like when you're shooting a movie, you don't just say do it all in one shot, right? Unless you are a very talented director and actor. You know, you have mise en place, you have staging, you have blocking, you have cinematography. It takes a while to composite the scenes together. This will be the same for this, but a large part of it will then be automated for creating the story that can resonate with you. And then you can turn it into Korean or whatever. And there'll still be big blockbusters like Oppenheimer and Barbie, but again the floor will be raised. Overall, similarly, we had a music video competition. Check it out on YouTube with Peter Gabriel. He allowed us to use kindly his songs and people from all around the world made amazing music videos to his thing, but they took weeks. So I think there's somewhere in the middle here where again, we're just at that early stage because ChatGPT isn't even a year old. You know, Stable diffusion is only 14, 15 and they're.

And ne. And I think you'd agree that neither of them is the end all and Be all. It's just, it's the earliest days of this field.

I had the conversation, the tiniest building block.

Yeah, I had this conversation with Ray Kurzweil two weeks ago. We were just after a singularity board meeting. We had, we just hung on a zoom and chatted. And you know, the realization is, unfortunately, the human mind is awful at exponential projections. And despite the convergence of all these technologies, we tend to project the future as a linear extrapolation of, you know, the world we're living in right now. But the best I can say is that we're going to see in the next decade, all right, between now and 2033, we're going to see a century worth of progress. But it's going to get very weird very fast, isn't it?

I mean, there's two way doors and there's one way doors. Right. Like in December of last year, multiple headmasters called me and said, we can't set essays for our homework anymore. And every headmaster in the world had to say that same thing. It's a one way door.

Yes.

And this is the scary part, the one way doors, right. Like when you have an AI that can do your taxes, what does that mean for accountants? All the accountants at the same time. It's kind of crazy, right?

It is. And the challenge, I mean, one of my biggest concerns. So listen, I'm the eternal optimist. I'm not the guy who's the glass is half full, it's the glass is overflowing. And one of the challenges I think through when I think about where AI AGI, asi, however you want to project it to, is the innate importance of human purpose. And unfortunately, most of us derive our purpose from the work that we do. You know, I ask you tell me about yourself and you jump into your work and what you do. And so when AI systems are able to do most everything we do, not just a little bit better, but orders of magnitude better, redefining purpose and redefining my role in achieving a moonshot or a transformation. It's the impedance mismatch between human societal growth rates and tech growth rates. What are your thoughts there?

Yeah, I mean, I think again, exponentials are hard. Like if I say GPT 4 in 12 to 18 months on a smartphone, you'd be like, well, that's not possible. Why? You know, like GPT4 is impossible. Stable diffusion is impossible. Right. Like now they've almost become commonplace. But why would you need supercomputers in these things? So I do agree there's this mismatch. And that's why we're in for five years of chaos. That's why I called it stability. I saw this coming a few years ago and I was like, holy crap, we have to build this company. And now we have the most downloads of any models of any company, like 50 million last month versus 700,000 from Istral, for example. And we will have the best model of every type, except for very large language models by the end of the year. So we have audio, 3D, video code, everything, and a lovely, amazing community. Because it's just so hard again for us to imagine this mismatch. There's a period of chaos, but then on the other side, like there's this P. Doom question, right? The probability of doom. I can say something. With this technology, the probability of doom is lower than without this technology because we're killing ourselves. And this can be used to enhance every human and coordinate us all. And I think what we're aiming for is that Star Trek future versus that Star wars future.

Yes, I'm into that and I think that's an important point. That the level of complexity that we have in society, we don't need AI to destroy the planet. We're doing that very, very well, thank you. But the ability to coordinate. So one of the things I think about is a world in which everyone has access to all the food, water, energy, healthcare, education that they want. Really a world of true abundance, in my mind is a more peaceful world. Why would you want to destroy things if you have access to everything that you need? And that kind of a world of abundance is. Is on the backside of this kind of awesome technology.

We have to navigate the next period. I believe we'll see it within our lifetimes, particularly if we get longevity solved. Right. And that's so amazing. Right? But then we think about, as you said, why peace. A child in Israel is the same as a child in Gaza. And then something happens, a lie is told that you are not like others and the other person is not human like you. All wars are based on that same lie. And so again, if we have AI that is aligned with the potential of each human that can help mitigate those lies, then we can get away from war. Because the world is not scarce. There is enough food for everyone. It's a coordination failure. And that can be addressed by this technology.

Agree. One of the most interesting and basic functions or capabilities of generative AI has been the ability to translate my ideas is into concepts that someone who has a different frame of thought can understand.

Right, that's, that's what this generative AI is. It's a universal translator for sure. It does not have facts. The fact that it knows anything is insane. Hallucinations is a crazy thing to say. Again, it's just like a graduate trying so hard. GPT4 with 10 trillion words and 100 gigabytes is insane. Stable diffusion has 100,000 gigabytes in a 2 gigabyte file. 50,000 to 1. Compression is something else. It's learned principles, yes.

It's knowledge versus data.

It's knowledge versus data. And you apply some experience, you get the wisdom, because it's learned the principles and contexts and it can map them to transform data. Because that's how you navigate. You don't navigate based on logical flow. We have those two parts of our brain navigate, sometimes based on instinct, based on the principles you've learned. Tesla's new auto driving model, self driving model is entirely based on, I can't say which architecture. They said it publicly is based on this technology. It doesn't have any rules. It just learned the principles of how to drive the massive amounts of Tesla data that now fits on the hardware without Internet. And so they went from self driving being impossible to now, hey, it works pretty well, you know, because it's learned the principles. And so that's why this technology can help solve the problem. This is why it can help us amplify our intelligence and our innovation. Because it's the missing part, the second.

Part of the brain, you know. Next. I can't give more details yet, but next week we're announcing the largest X prize ever. It's $101 million. It's 101. So it's Elon had the, had $100 million prize that got him to fund a few years ago for carbon sequestration. And the funder, the first funder of this prize wanted to be larger than Elon's. I said, okay, you add the extra million.

That's for luck. It's for luck. We did our seed round at 101 million.

Oh really? Okay, that's great. That's a popular number anyway. The. And it's in the field of health. I'll leave it at that. Folks, go to xprize.org to register to see the live event. On November 29th, we're going to be debuting the prize. What it is, what it's going to impact 8 billion people. Long story short, it's, it's a nonlinear future because we are able to utilize AI and make things that we're seemingly crazy before likely to become inevitable. And it's an amazing future we have to live into.

Yeah. I mean again because it's one way doors. The moment we create a cancer GPT this is something that we're building. We have trillions of tokens and Google TPUs and things like that that organizes global cancer knowledge and makes it accessible and useful. Even if it's just for guiding people that have been diagnosed. The world changes. The 50% of people that have a cancer diagnosis in their lives in every language and every level will have someone to talk to and connect them with the resources they need and other people like them and talk to their families, you know and how insane is that? And so I can. These positive stories of the future need to be told. Right. Because that will align us to where we need to go as opposed to a future full of uncertainty and craziness and doom.

In our last couple minutes here buddy, what can we look forward to from stability in the months and years ahead?

We have every model of every type and we'll build it for every nation and we'll give back control to every nation.

So coming back to governance here again is the nation state the unit of control, is it?

No. My thinking is the stabilities in every nation should have the best and brightest of each. Because what you've seen is there are amazing people in this sphere. The best and brightest in the world know this is the biggest thing ever and they all want to work in it. And it's just finding the right people with the right intention. The brightest people go back to Singapore or Malaysia or others because of the future of their nations. And again that now we're doing a big change and we don't talk about all the cool stuff we do. We've just taken it because you need to articulate that positive vision of the future. Because the only scarce resource is actually this is human capital. It's not GPUs, it's not data. It's about the humans that can see this technology and realize that they can play a part in guiding it for the good of everyone, their own societies and more. And that's again what I hope stability can be.

Well, I wish you the best of luck pal. Thank you for joining me in this conversation. It's. It's been a crazy four or five days and wish Sam and Greg and the entire OpenAI team stability in their lives.

I hope they have a nice Thanksgiving. They're an amazing team. Building world changing technology. It's such a concentration of talent. I think again. I really felt for them over the last few days, much as I kind of post memes and everything, posted that as well. I think this will bring them closer together and hopefully they can solve the number one problem that I've asked them to solve, which is email. Solve email. Right. And then we'll crack on from there. All right.

Cheers, my friend.