Taking Seriously The Threats Posed by A.I., with Tristan Harris

This is Larry Lessig. I got an email out of the blue from someone I didn't know, someone who's been following my work or the work of democracy reform. And this email asked me what have I been doing the last couple years? And it was kind of a punch in the gut, because the reality is, in a way that sometimes I regret I've done nothing over the past 16 years except figure out how to advance and to work as hard as I could to advance the fight for democracy reform. But the reality is, the conversation about democracy reform tapered off pretty dramatically after the extraordinary defeat last year in January in the United States Senate of the Freedom to Vote act, which was the chance we had to make fundamental democratic reform possible. And since that time, I confess there was a period where it seemed so depressing that I wasn't doing much publicly, though privately, I was doing a ton of writing and scheming and conspiring and planning and doing all the things one needs to do to continue a fight that just seems like it will never end. Okay, so here's the next chapter. This is a preview of the next season of Another Way, the podcast, which I imagine you're listening to because you chose to listen to this podcast, but maybe you were forced to listen to it, I don't know. But anyway, this is the podcast Another Way. We've had a number of seasons. This is a new one, or it will be a new one. I have, over the summer, recorded. I think I'm up to 15 now. Separate interviews with a wide range of people touching many different issues, all to be wound together into an argument about where we are and where we need to go. And that argument will eventually be a book. So this podcast, which will be released beginning in September, is the foundation for the book that I will be writing over the course of the year. And you can get it either through the podcast or through the book or through both, whatever you want. But there were two interviews that I've done in preparing for this season that I want to release in advance. I'll also release them as part of the season because they fit in a certain order and there's a certain story I'm trying to tell. So I will plug them in there and you might skip over that part when you get back to it. But these two interviews are powerful and important right now, long before we get the final editing done. And so I want to release them in advance. And the first I'm releasing today is an interview with Tristan Harris of the center for Humane Technologies. And this is A conversation with him about the threat posed by AI, a threat to our political culture, our social culture, but importantly, as you'll hear in this interview, a threat we've already had, as he so nicely put it, first contact with. And when we look at what we did after that first contact with that threat in the context of social media driving through its business model of engagement to get us as engaged as it could, when you look at what we've done in response to that threat, you'll see we've done actually very little. Which is the biggest reason to be fearful about AI going forward. Because the next round of AI in the context of this next political election is going to be orders of magnitude more significant in affecting how people engage or don't engage, how they think or they don't think, how they understand or trust or don't trust this democratic process. I've been working with Tristan for a number of years. Let me just introduce who he is and where he comes from. Tristan Harris is the executive director and co founder of the center for Humane Technology. Before that, he was a design ethicist at Google. He went to Google after receiving his bachelor's degree from Stanford University, where he was a student of the famous BJ Fogg, who ran the persuasive technology lab at Stanford. In his role at Google, Tristan began to have a recognition that the technologies that he was helping to build were not just passively connecting to society, that these technologies were changing society and they were being designed to change society by changing individuals. And that's what led him to step away and to begin to critique the way in which this design architecture was affecting society. And that critique expressed itself most powerfully, powerfully, in perhaps the most successful documentary in the history of documentaries, in the sense of one that's been seen the most and seen many times by many people. The film the Social Dilemma, which brings together a number of people from Silicon Valley to tell the story about how the industries of Silicon Valley architect their products to hack our brains to get us to engage with their platforms. These are not evil people who are trying to destroy us. These are Silicon Valley engineers and entrepreneurs trying to find a way to make a product that gets us to connect enough so that they can earn the ad revenue that they seek. Now, most of his work has, until recently, focused on the question of social media, and certainly there's a lot of work to focus on there. One of the people that I will be interviewing or I have interviewed in the context of the series, Jonathan Haight from nyu, is going to Talk extensively about what we now know about the harm that comes from the way social media is being used. But more recently, Tristan has become focused on the threat presented by artificial general intelligence and AGI technologies, which are increasingly becoming accessible and used in this extraordinary competition between big actors trying to gain market share so they can control the future of this innovation. As you'll hear in the interview, Tristan was first skeptical of the great fear that many have about how AI will interact with society, but he became convinced that this is an existential threat. And through the course of the conversation that we have over this hour, you will hear exactly how he came to see the threat and how we should understand and resist the threat. Okay, again, this is one interview in a series that will launch in September around the arc of the book that I'm developing, but it's a preview to that book, separate from the argument in that arc, for the purpose of spreading the ideas in this conversation as quickly and as effectively as we can. Tristan, it's so great to talk to you. You're maybe the person that people listening to this podcast will know most clearly. You were prominent in the extraordinarily successful documentary the Social Dilemma, introducing to many people for the first time the great threat that social media presented. But we're going to have a conversation about AI and I want to start in the context of social media, because people talk about AI as if it's something in the future or we've just touched it now that we've seen ChatGPT. But. But your work in the context of social media has all been about the consequence of AI that we already are living through, and we already see the harm from that. So why don't you just introduce that in the way that I know in 50,000 talks you've introduced it, but that you feel most comfortable getting people to understand exactly what the thing is we should be fearful about.

Yeah, sure. Well, first, it's obviously always an honor to be speaking with you. And so, I mean, what are we worried about when we're worried about AI? Well, oftentimes you study the sci fi books. It's like, well, what if we give it a goal and that goal is somehow misaligned with humanity and it races to pursue that goal in such a way. But then, don't worry, we would notice the misalignment with AI and then we would fix and correct it, or we would unplug the machine if it was ever causing damage. Well, we're here to sort of say that humanity already went through this scenario with social Media, because this was a curation AI. You open up TikTok, Twitter, Instagram and you flick your finger up on the screen to say, what's the next video TikTok's gonna show me? And you activate a supercomputer, point it at your brain, narrowly optimizing for this narrow misaligned goal of engagement, which is what is just the thing that I'm making a calculation of which video could I show you next that would keep you watching? And that just very simple curation AI that's just picking which of all those billion videos could it show you? Just picking the one video to show you was enough to unravel shared reality of democracies, drive a mental health crisis for youth, shorten attention spans, drive up polarization, cyberbullying, teen suicides, like all of these things, this climate change of culture from this misaligned, this very narrowly misaligned AI that's just picking videos to show you or tweets to show you or what have you. And notice, have we fixed the misalignment with social media once we noticed that this goal was misaligned? No, we didn't fix it. Did we unplug the machine? No, we didn't unplug it. And in fact, you could even ask Mark Zuckerberg or the guys who run TikTok, aren't there harms with AI? And they'll say, no, I'm an optimist, I really believe in the goodness of technology. It's like, well, yeah, tell that to the Maria Resta in the Philippines or the people in Myanmar whose countries have been genocided by these misaligned AIs. So the reason we say this is that that was first contact. We like to say social media was first contact at scale between humanity and AI being narrowly misaligned and then not fixing that misalignment. And important to diagnose in that was not that anyone who was at these companies had a bad intention or wanted any of these horrible things to happen or wanted to harm democracy. They simply were wired up to this business model of attention and engagement because of the advertising business model. And this was the consequen. Now as we're heading into second contact between humanity and AI, which is not curation AI picking which content to show you, but creation AI generating content at scale, the ability to generate text for law, the ability to generate media at scale. Videos, photos, text, code. Our world runs on language. And this next AI is run on the ability to manipulate, synthesize and decode language at scale. Important to say that what was driving the social media AI problem was an arms race. Except it was an arms race to this narrow goal of engagement. If I don't do it, I'll lose to the guy that will. So that logic created the race to the bottom of the brainstem, which is a perverse incentive where those who are best at hiding the externalities of, yeah, sure, we increased engagement for Snapchat, but we did it by basically ruining a generation's mental health, just so our 401s went up for everybody who had stock invested in the Snapchat companies. So that was the game theoretic dynamic that was driving social media. And importantly, we never said, oh, there's just this dark side to social media. There's a. There's a dark side to a race to a perverse incentive. So let's really take the game theory in here. Okay, now switching to second contact.

No, no, I don't want to switch it because I want to make sure we unpack the first stage because I know that you know everything you said. I shaking my head in violent agreement, but I know that when I try to explain this to other people, the steps are missing in their head. So let's unpack a little bit and let's go back to a very ancient technology. Let's talk the Facebook News feed. The Facebook newsfeed, which was created pretty early, but only really became something about news relatively late in the arc of the life of Facebook, is deciding, or it's deciding. The AI is deciding which content it's going to serve me. What is it looking at to decide which content it's going to serve me? What kinds of factors are important to it?

Well, it creates. It looks at clusters of user behavior. So of the whatever, 2, 3 billion users of Facebook that exist now, it says, okay, you look like a person who's clicking on all of this QAnon content or this Trump content or the antifa content or name your political tribe. And the kinds of things that get you to keep scrolling are why Biden is so bad or why Trump is so bad, and whichever tends to keep your group scrolling. Like the supercomputer that powers the Facebook newsfeed is just looking at patterns of user behavior. And it's already seen, before I scroll to the next thing, it's already seen a million other users who've already also clicked on and liked and outraged and commented on the thing that Trump did yesterday that we should all be angry at. And so it just knows that that thing that made those other people Angry is also going to make you angry. Now it doesn't know anger really. It just knows what's engaging. It just. You know, there's this book, seeing like a state, right? There's sort of analogy of seeing like an AI. If I'm seeing like an AI, all I know is one thing which is just what is driving up. Likes, clicks, engagement, views and shares. Those are the only eyeballs that are plugged into how this AI is sorting what material it should get. People might quibble with that. They'll say Facebook does long term surveys on user well being, blah, blah, blah. Yes, they do a little bit of that. But really fundamentally the, the business model is engagement. And that was enough to basically identify these clusters of users, give them different pieces of content that would satisfy that specific tribal group and that would start to create the speciation inside of reality, these islands of reality that we get very different kinds of material. And even going into the 2020 election, it was shown in an October 2020 study, there's an MIT Tech Review article about this, that something like 140 million Americans per month were being reached by Eastern European troll farms. That actually the top 15 out of 15 Christian groups on Facebook. So reaching Christian Americans, all the top 15 groups were run by Eastern European troll farms. And these are people who are gaming the Facebook group's recommendation algorithm. Because you join one Christian group and there's some product manager at Facebook whose job is also to do what to optimize for engagement. Which means more people joining more groups, which means let's make that prediction engine for which Facebook group to recommend to you be more and more accurate and successful at causing you to hit the join button. But causing you to hit the join button on a group doesn't distinguish between who are the people behind that Christian American group. And the top 10 out of 15 African American groups on Facebook were also run by Eastern European troll farms. And we are not talking about a 2016 scenario here. We are talking about a month before the November 2020 election. So now here we are after Frances Haugen's done her whole thing and a lot of people have left Facebook. It's not really cool to work there. So you have the junior varsity team sort of chipping in now as we're heading into the 2024 election. We don't have nearly as many people working on the trust and integrity of these systems. And the fundamental business model is really misaligned. They can add some more whack a mole sticks. And yeah, sure, we should want Facebook to be Hiring more teams with more capable people with higher quality. Whack. A mole sticks. But also, that's still denying the fundamental code, the DNA of what Facebook is about and what Twitter is about and what TikTok is about, which is engagement. And that was the first contact problem.

Right now, many people, when they think about how the machine is actually doing this, have in their mind a machine that has tons of private data about individuals that it's kind of in some sense and accessing to figure out which thing I should do next. But the way you've described it is it's not so much that there's private data sitting around there, as the machine has learned a model, it's built a model around you. And it might be that there are 69,000 different kinds of people in the world and only 69,000, like, we are not so different. And that once it figures out you are a person, 24,448, it knows exactly what it should be serving you without keeping any of your data. There was a certain point in which Mark Zuckerberg said he was happy to get rid of all the private data, but that was because he didn't need the data. That privacy wasn't really what was driving this. It was what the machine was learning that was then the engine of producing this harm. Is that right?

Yeah, that's exactly right. I mean, I think a lot of people want to focus on the private data that Facebook stole or robbed from you. But as you said, I can probably, if I'm Facebook, I can throw away a lot of the private information and just use what you click on, what you like, what you tend to engage with, how many seconds you watch certain kinds of videos to predict certain things about you. And a friend of mine used to be the sort of senior vice president of advertising at Facebook, and he assured me there's this conspiracy theory that Facebook is listening to the microphone. This is at least a few years ago. Maybe they're doing something now. But a few years ago, there's this conspiracy theory. People say, oh, it's listening to the microphone. How do I know that? Because I was just having a conversation with my friend about some topic, and I open up my Facebook newsfeed and there is the exact shoe brand that I was just talking to my friend about. So surely it has listened to me on the microphone. And this friend of mine who ran advertising at Facebook said, I can assure you, Tristan, as like, just full, good faith, honest, mano a mano conversation, we do not do that. And there's Forensics that show that they do not do that. The truth is even more alarming, which is that they don't need to listen to the microphone. They just have enough data points based on everything that I tend to click on that, just as you said, I look like, you know, one of the only 5000 kinds of user groups on Facebook and one of those groups tends to talk about the. Or like these kinds of shoes. And so that's maybe even more alarming.

Yeah. So, but so then when when AOC entered into the TikTok debate and she did her first TikTok and she said that we should not be like, regulating or banning TikTok because we actually weren't protecting privacy for American companies. So until you're protecting the privacy of Americans for American companies, we shouldn't be worrying about foreign companies. That was missing the point. Right. I mean, we should not be against privacy regulation. Of course, we need good privacy regulation. But even with privacy regulation, we still are not addressing the problem you're talking about.

That's right. Yeah. It would help to have privacy regulation because, sure. If I have even more data collected on you that Facebook buys private data on you from a thousand other providers to. To build an even more accurate profile that they can harvest even more money per user, would it limit some of the revenue that they get from you? Sure. Can we quantify that? I'm not sure. I don't personally know. It's hard to sort of say exactly. The issue with TikTok is a lot more simple. If you go back to the Cold War, would the United States have allowed its number one geopolitical competitor, the Soviet Union, to run television programming for the entire Western world?

We know the answer. That was illegal. You couldn't have a foreign company owning a television station, period.

Right. And now we have literally outsourced the number one most popular social media app, and it's choosing what everybody sees and what some people don't know is inside of China domestically. ByteDance, the company that owns TikTok, has a different version of TikTok for its Chinese population called Douyin. And effectively, to use a metaphor, they feed the digital spinach version of TikTok to their population. When you open it up, you get financial advice, patriotism, videos, literally videos about who won the Nobel Prize. Quantum physics. What is quantum? Quantum physics. Literally. I had a friend, a Chinese tech entrepreneur, he showed me on his phone, there was the two TikTok icons, and he opened up the Chinese one, and it was literally videos about who won the Nobel Prize. And Then Xi Jinping and so on. And then we opened up the American version, just the regular international version, and we call it the digital cracker fentanyl version. Right. It's basically just the mindlessness that keeps people engaged. Now in saying this, am I saying that the Chinese Communist Party deliberately from the beginning engineered this mass psyop to just kind of break America? No, it was just this really successful, successful engagement company that won the race to the bottom of the brainstem for engagement. TikTok simply did even more addictive videos, even faster clip with a tighter sort of swiping slot machine kind of dynamic. They dialed that thing in even stronger. Now, once that thing really won in China domestically, they regulated how TikTok worked to be more like digital spinach. But they're happy to ship the kind of reckless version to the rest of the world. And honestly, as I said in 60 minutes back in November of last year, right now, the number one most aspired to survey that was done a little while ago. If you're in the United States, the number one most aspired to career among young people is a social media influencer. And the number one most aspired to career in China is astronaut. That's kind of all you need to know. I can just walk away from your population and 10 years later I kind of know which society is going to be successful. So this is kind of a no brainer. There are some decisions that are hard, there are others that are easy. Banning TikTok is a very easy decision on national security grounds alone. Now to others points they will say that doesn't solve the problem of the engagement economy and they're absolutely right. We then have to go tackle the race to the bottom of the brainstem and the engagement perverse incentive to change that incentive to something else. And that's possible. You could say that any information system, technology, information system that is managing the information commons of a large enough percentage of a democracy must not maximize engagement and they have to be instead optimizing for a different goal that can be democratically deliberated, which is something like the healthy information commons for democracy. So you could rank for example for unlikely consensus as they do in Taiwan. So you could have say Twitter and Facebook and TikTok all be required. And I'm jumping to the conclusion here, which is I think that a democratic deliberation of citizens thinking about what should they optimize for would land at something like. And just take this as a one possible solution, unlikely consensus, which means you rank for content that has positive sentiment from across different, tending to be disagreeing political tribes. And whenever you get a rare and unlikely consensus with positive sentiment, you rank for that kind of content. They're called bridging systems. If we had that Running, Twitter, TikTok, Facebook, et cetera. How quick? As fast as Facebook makes people angry, lonely, scared and polarized. I'm not saying techno utopian, it would fix it all, but it would shift the direction of where this is all heading. And that's a totally plausible thing to do. That's different from privacy legislation and it's separate from. But in addition to banning TikTok, now

when you think about the regulations that actually exist in the context of China, I mean, it's important to recognize the two versions of TikTok are not just two platforms, it's also infrastructure for access. If I'm a 14 year old in China, how am I allowed to use technology?

Yeah, I think what you're referring to here is that they just also have different rules for young people in China. So in China, as I understand it, they have the cyberspace administration of China, which is the largest regulatory body in China is for technology, is for cyberspace. So that would be like in the United States we had some very, very big regulatory body that was all about cyberspace. And one of the things that they regulate is effectively time, manner, place, type restrictions, but for usage. So if you're, I forgot the age. If you're under either 14 or 18, but you can only use social media for something like 40 minutes a day for gaming, you can only do games on Fridays, Saturdays and Sundays. And importantly, an easy one that I like to talk about is from 10pm until I think at 6 or 7 in the morning, it's lights out. So in other words, if you open up social media, like you literally see the TikTok app on your phone in China and it's 11pm and you open it up, it just literally says it's closed and it's not gone. It's not that it disappeared or it's blocking you forever. It's just we're going to open back up at 7 in the morning. It's, it's just like opening hours and closing hours or just like a weekend. And what that does that's so smart is that a lot of kids get sucked into the dynamic because their friends keep talking to them at 11 midnight one in the morning, they tag them in. Another thing, they might gossip about them and they feel the social pressure that if I don't keep using it till later and later and later at Night, I'll miss out on something. And that really eats at kids. And so what they did was not allow kids to use it. They just simply limited the times. It's just like weekends. You could technically take a hyper libertarian view and say weekends are this horrible authoritarian thing that you're forcing everybody to take the same two days off. If we want to go hyper libertarian, everyone should be able to split those 48 hours anywhere they want to in their week. And that would be better for the individual. But what we really want is to all have the same two days off at the same time so that we get the benefits of coordination. And the Sabbath and the weekend are a kind of coordination technology. And the lights off from 10pm to 7 in the morning is another kind of coordination technology.

Yeah, I spend a lot of time in Germany, you know, in the early, early 80s where they had really strict Sunday laws and it was so peace inducing. There were like days where you just go out for a walk and you'd have a picnic and like there was nothing else to do. You couldn't even go to the grocery store. And, and the point is it was about creating a rhythm in life. Now it's one thing to argue you should do that for adults, and I think many Americans would resist it for adults. But what's striking is how resistant Americans are for to laws that would even do it for kids. I mean, you know, it's a controversial thing to say. Why don't we just have opening hours for these technologies? And the first response I've found, and I'm eager to hear the interactions you've had because you've obviously been doing this much more extensively. But the responses I've had is, first of all, it can't be done technically because you can always find a way to hack around it, which might be technically true, but not 99% of people are going to find a way to hack around it. And secondly, it's just unfair or it's against the idea of capitalism or the free market. And what's so striking about that response is it's like, well, I mean, we don't sell kids cigarettes and we don't sell kids alcohol. And that doesn't seem to end capitalism as we know it. So what is the issue here?

So first of all, could people work around it? I think when people think about that, they think about things like the Apple screen time controls and the ways that kids work around those things. In China, as I understand it, literally the app itself, the company is required to have it go dark so there's literally no way to work around it. There is no workaround. There is no. The kids sneak past the parents, parental controls. So it literally just does work. That's the first bit. The second question you said again was.

Yeah, I mean, the idea that we can't be limiting in the context of kids is of course, belied by the fact that we do limit in the context of kids already. Like, we have very clear sense of what kids should not be able to do. Yeah. And why. Why it's so hard to extend it here is just kind of baffling. And the only states that are attempting to do something close to this, nobody's doing close opening hours, as far as I know. But states that are doing it are red states, which is, I think, very troubling, because if it becomes politically polarized, then obviously that's a way to stop it from happening anywhere.

Yeah, I mean, you're referring to a few different things here. I mean, one of the reasons I worry that TikTok is not being banned is that some politicians on some sides of the political spectrum have now gained an advantage in, let's say, playing the TikTok game better than their opponents to win elections, which is, you know, people worry about, you know, Donald Trump worried about the 2020 election being hacked and stolen because they were manipulating the vote. Well, if I'm literally like the Chinese Communist Party and I'm running the information information system that determines what you're going to see going into the next election year, we should be much more worried about TikTok being the mass influence over what people believe. Not, by the way, to cause people to vote for someone other than who they're probably already going to vote for, but to change the overall sentiment and what we believe about each other and the level of divisiveness in society, let alone the fact, by the way, and this is the argument I gave to Senator Warner and those who have been sponsoring the restrict act to ban TikTok. If China were to invade Taiwan tomorrow, how would the world know whether the United States or China started it? They would look at social media. And what if on TikTok the day that that happened? I'm the Chinese Communist party, I use TikTok and I amplify domestic voices in your own country. I'm not creating new propaganda saying the US Started it. I'm amplifying only the voices in the US who say the US Started it, because that's how they view the world. And then I amplify those voices 101 compared to the other ones. And now it's not propaganda, it's amplifaganda, selective amplification of the views that I want. And you have literally given me the control, the remote control to your entire political system and you're not turning it off. And if I'm the Chinese Communist Party, I'm laughing all the way to the bank at the lack of technical competence in your society to make a very, in my mind, very simple choice. The issue with kids, I think, and the belief that this is just capitalism is, I mean, you already gave the arguments we don't sell. There are limits for alcohol, cigarettes, pornography, et cetera, to certain ages. I think the thing that people have to see here is we have to be careful about the language we use when we say that kids are showing us their revealed preferences. This language of revealed preferences is a very obfuscating language. If I said I wanted to go to the gym or I wanted to be with my friends in the pic to have a sunny day in the park, but then I end up scratching, scrolling TikTok for three hours, according to this, you know, if I use the phrasing of revealed preferences, then the fact that I'm still scrolling those videos is my own choice. That that was the truest view of what I really wanted. And it was just bullshitting when I was saying earlier that I wanted to be in the park. But really what's going on is I have a supercomputer pointed at my brain that's literally calculated what a million other humans have watched next that definitely kept them scrolling. And I'm making more and more perfect predictions every day because. Because as they make more money, TikTok, where do they reinvest their profits? Where do they reinvest their revenue? Into more data centers, into a bigger supercomputer. And to calculate even more accurately next time and gain an even greater asymmetric advantage in predicting the perfect videos that I'm going to show you. So we have this runaway asymmetry of power that on one side of the screen there's just your brain and the other side of the screen there's a supercomputer that's getting stronger and stronger and more powerful literally the more days and seconds that that people use it. And we have to start looking at things in terms of asymmetries of power. Worse than that is that these companies have colonized the meaning of social participation. I could be a 14 year old and say, you know, Larry and Tristan, I completely agree with you. I will never use TikTok but then guess what, let's say I go to a high school where my only friend, my friends all use TikTok to just to send a message back and forth. So like what? Texting is for you and I, we can send a message to each other on signal. We can participate in society that way. Let's say that this 14 year old's friends only use TikTok or they only use Snapchat. You can't extract yourself without socially excluding yourself from some of these social media platforms. And that's another aspect of the asymmetry of power and the way that these companies game these ideas of revealed preference.

Okay, one final thing before we move to chapter two or second contact, many people will say, look, these companies don't want to hurt people and once they begin to see the harm, we should expect they're going to self regulate to avoid doing the harm. Now you and I have you more than I, but we both had a lot of exposure to the Facebook files, which the most interesting thing I drew from the Facebook files was the incredible integrity of many of the engineers inside of Facebook who were trying to do the right thing, who would like point out problems and point out potential solutions that would address those problems. And time and again they were interfered with because of, as you nicely put it, the business model that the company has. So what is the possibility or the plausibility that these people could be talked into, quote, doing the right thing in a context where the business model is contrary to that?

Yeah, unfortunately, they just can't be talked into it. And we should fool me once, what is the trust? Fool me once, fool me twice. The point is you can't phone me again. I think he says the third time

around, what's great is that we all make fun of him for not knowing it, but none of us know it either.

I don't know it either. So it's all fine. And that was not a political statement. I was just joking. So yeah, I think in the Facebook files, Francis Haugen in one of the documents, when they were choosing to optimize for something called meaningful social interactions or time well spent, which is actually Facebook kind of took a concept that we were incubating and publicizing and saying we're not going to rank to maximize time spent, we're going to maximize time well spent. But even in Frances documents, and I don't remember where this was, but there was some example she was telling me about where if ranking for time well spent would reduce engagement or time on site by more than 1%, even 1% was enough that they wouldn't compromise because again, they're also competing with TikTok and Instagram. And by the way, the more successful TikTok gets compared to Instagram and Facebook, the more they're going to reduce the investments that they've already made in trying to make it better. Because at the end of the day, it's the competitive dynamic that's going to drive what Facebook or TikTok do. They'd all like to do better, just like you said. And you and I both have friends who worked at these companies and who work really hard, by the way. And we should applaud the hard work and thankless work that a lot of people, people who work on trust and integrity or work on election security or all these kinds of teams that Francis worked on as well. And it's really thankless work. It's very hard psychologically and people burn out after a couple years. And it's very sad and they deserve this applause. But the problem is the things that they would like to see that would structurally fix the problem will not get adopted. And that's what we also found in the Facebook files, that people had recommendations on the trust and safety teams about how to make things safer. But in many cases those were not adopted because they would conflict with that engagement business model.

Okay, there's so much more we could talk about, but I want to help people understand how you see the great threat to second contact like, and this really nice way that you framed it about the difference between them. Just looking at our revealed behavior and modeling on it versus them. Giving us content itself, I think is a nice way to tee it up. But. But when you first started recognizing what was happening in this place, what were you most worried about? What was the outcome that you're most worried about? And what is the business model that would be driving that outcome?

Yeah, this is important because it's not immediately apparent what business model incentive is going to drive the harm. In the case of generative AI, think about ChatGPT. It's a blinking cursor. It's just asking for me to ask it a smart question and then give me an answer that seems pretty harmless to most people. So what's all the fuss here? And Sam Altman can say it's just a tool. It's not agentically pursuing anything. It's a blinking cursor. You have to type in what you want. It all sounds very, very convincing. And I want to steel man that perspective first, because I think that is how a lot of people relate to it. I mean, I have friends who are Stanford PhDs in Philosophy and Stanford Law School doing dissertations, and they came up to me at a party a couple weekends ago and they said, tristan, I'm really thinking about this, but I just don't see the risk with AI. And so I think we should really honor that. It's not immediately visible. And I'll name, by the way, I used to occasionally go to these events or conferences where I'd meet folks in the effective altruism community who were working on existential risk around AI. And I thought it was overblown. First of all, I want to say I thought it was sort of sci fi. I actually thought these guys are missing the fact that we already have a runaway AI called social media and we're not even seeing it because it's not called AI, it's called social media and we're kind of missing it for a simple ontological error. So I want to first say that I did not come into this wanting to be an AI doomer or a existential risk sci fi nut. So, first statement, second statement. Back in January of this year, January, February, I got some phone calls from people who worked inside of some of the artificial general intelligence labs, basically hinting that the arms race was about to jump a huge step forward with the coming launch of GPT4 and that this was moving at a pace that was very, very dangerous, that this was not ready to deploy to the public, that we shouldn't be moving at the speed that we're currently going, but that the arms race had now been kicked off because Microsoft had pushed OpenAI to launch ChatGPT and then they were going to integrate GPT into Bing and all these things and, and that for profit. Arms race to launch those new generative AI systems into Microsoft Office. Bing, the Bing taskbar was going to kick off this market dynamic. And so back to your question. What is the actual business model that's going to drive the harm here? And it's not really the business model so much as it's the model of the race for dominance. So if I'm OpenAI, how do I win? I have to get everybody using my AI platform and not using the other ones. I don't want people using Google Bard, I don't want people using Anthropic's Claude chat agent. I want them using OpenAI. So what do I want to do? I want to open up. I want to first launch first, right? And then I want to ship the bigger, more capable model as fast as possible so that people start getting benefits from it earlier. Then I want to build a developer platform that's opened up for everybody to use. And if I do that faster than the other guys, then everyone's going to start building on my thing. So I'll get startups building on OpenAPI, OpenAI's GPT4 API. I will get a government's building their next systems on top of my API. So the race for dominance translates into a race to deploy as fast as possible. But the race to deploy translates into a race to recklessness. Though those who ship faster and without as much safety testing will win over the ones that say shoot. Google, for example, as my understand, it had a bunch of advanced AI capabilities in synthetic media. The ability to synthesize faces, video, text, et cetera. But they held that back because they knew that that wasn't really safe for the public. But let's say someone like Stability AI races into the picture and says, we're just going to hand these tools to everybody and you can run it on your own laptop. Now Google will simply be the billions of dollars of investments that they've placed into those capabilities. They're going to look pretty dumb if they don't put them at least out there in the world so they can stay in the game. And it's really not evil people or bad people. I've met so many people work at the AGI companies. There's a lot of people very concerned about safety and risk, which I should say by is different than the case of social media. Social media companies did not start or were founded with the notion of how much risk was involved. Imagine if Facebook had started with a Facebook risk team that they knew that they could basically destroy society With Facebook from the very beginning and they had teams that were thinking about that. That didn't happen. But with AI companies, almost all of the artificial general intelligence companies started with these AGI risk teams where they knew that they had the power to really cause catastrophes in society. And so they've been thinking about it for a long time. Those people are working very hard. But again, the overall prevailing thing that is going to print the pages of world history are these competitive pressures which now that they have been kicked off literally the people inside the safety teams of the individual labs don't know what to do to stop it. And so back to the story. After we got these phone calls in January or February, they asked us because we had the success with the social dilemma and had a public platform, could you use your platform and Relationships and institutions, institutional connections to try to prepare the public and activate institutions to help slow this thing down a little bit. And some people might not like the language of slow this down, because immediately they think, well, then what about China? China's going to race ahead. Why would we slow down? Let's ask a different question. How do we move at a pace that we can get this right? Are we moving at a pace that we can get this right? And the answer right now is no. The race is not to move at a pace that you just deploy plutonium everywhere in your society as fast as possible and say, well, we beat China because everyone has plutonium in our society. So now we're going to be able to harness the benef. But yeah, you also externalized a bunch of risk in nuclear terrorism because you weren't thinking about the actual races not to release plutonium. The actual races had to harness plutonium's benefits without the nuclear terrorism. And I think with AI, we are racing to deploy capabilities that are not coupled with or limited by the wisdom or responsibility that you would need to wield that capability, whether it's creation of bioweapons or AI that can hack legal contracts, or AI that can find cybersecurity vulnerabilities in code. So my own journey with this was really after we got those phone calls, we interviewed hundreds of people who work in safety at the companies and we asked them, what do you think people need to know about this? And my co founder of center for Human Technology, Azer Raskin and I, we spent hours on the phone with folks kind of taking notes and trying to synthesize what is the core issue here. And we put that in a presentation called the AI Dilemma, which we raced around Washington, New York and San Francisco and gave these three major briefings and invited all the top people that we know and then have since been talking to the White House. And I met with President Biden two weeks ago and trying to activate everybody we can to try to create better coordination. But the name of the game here is coordination, that when you create a new technology that confers power, if it confers power, it will start a race. Those who adopt that technology to gain a power advantage over others will start to use it. And if I don't use it and I'm against that technology, I'm just going to lose in that race. If you do not coordinate the race, the race will end in tragedy. With increasing power of the kind of technology that we're handing out, that tragedy can be a bigger collective tragedy than Simply the tragedy of I have a new fishing net that can maybe get more fish than the last kind of fishing net. So that tragedy is we fish out the commons, there's no more fish left. But when you have AI that you're deploying, the commons is really a social fabric that works, that you can think of society as having a finite absorption rate for new technologies, and you can overshoot by releasing more technology than society can healthfully absorb. Now, we've absorbed and been disrupted by new technologies before the printing press, the industrial revolution. We have to say that we have been able to adapt. And sometimes after decades of wars and so on that were caused by it, we can absorb new technologies. But this is the fastest pace that we've ever released one and last thing I'll say is it took Facebook four and a half years to get to 100 million users, and it took ChatGPT just two months to get to 100 million users. So if we're going at this speed and we haven't gotten into the harms yet or examples of it, you're guaranteed to not be able to get it right because we're moving too fast, that we can get it right, even if it looks good for right now. And we've seen other examples of DuPont chemicals. We're better living through chemistry. And here we just had a couple months, a month ago, the lawsuit from Forever Chemicals and PFAS that these are these forever chemicals that are in our environment and they never biodegrade. And there's a $10 billion lawsuit now for all of the farms in Michigan and Ohio that basically people have cancers and things like that, because we got this benefit of Teflon and these fire extinguishers and foam for flame retardants and these kinds of things. And it did offer these benefits, but we didn't look at the externalities, and it took us decades to find those externalities. In the case of AI, the externalities are going to come much faster, and AI is going to become entangled with many core parts of our society much faster. So the time to get ahead of this and to regulate this and put guardrails in place again, guardrails that create the coordination mechanisms for safety. It couldn't be more urgent.

So it's very helpful because the way you frame the initial threat is exactly the dynamic that happened early on with the operating system wars. Like, you know, the competition between operating systems. Why did Microsoft work so hard to make sure that Windows was the operating system? Because it understood that it would, that it would own something extremely valuable if all innovation was going to be done on top of that and it wanted nothing else out there. And when the threat was that the web could become its own operating system, they did everything they could to, to kill that potential. That was what the first Microsoft antitrust case was about. But the difference between that context, in this context is that the worst that would happen in that context is that your programs would crash or maybe you would lose your spreadsheet or your machine wouldn't run as fast as it would have run if you'd had a different operating system. Now, the externalities from that failure are existential at the societal level. And so one way to think, think about the problem here is to just recognize that in neither of these contexts was there an effective regulator. I mean, in both, in the Microsoft context, it took forever before the government recognized monopoly concern. But after they fought the Microsoft war, then they stood back and they allowed the Google generation of innovators to just be completely untouched by regulation. So governments have done nothing in that context. And in this context too, it's hard to imagine the government doing something important to intervene to slow things down. Now, is there regulation that you would say tomorrow could be passed that would say, here's how we're going to slow down, here's the speed limit for AI.

This is a big and complex answer, but I wanted to respond to a couple of things you shared there. So you're right that we. The race for operating system dominance, including the open architecture that was Windows, you can run it on any PC, whereas the Mac was only limited to the hardware of the Mac. And so the company that had the most open architecture won in the case of the AI wars. Now, Meta or Facebook is trying to play the open source AI cart. They want to be friendly to all the open source developers because that's the strategy for getting market dominance. But that means just releasing these models that are increasingly powerful, that can do increasingly powerful things. The last one they released, Facebook Llama, was not as good as GPT4, but people are tuning it to be better and better. And if the next llama comes out and let's say it can find cybersecurity vulnerabilities in code. Hey, find me the nuclear power plant, COBOL code or whatever it was, find me the water sanitation plant, find me the code for these things and let me hack it at scale, that's really, really dangerous. So just because something is open source and it's the best strategy for dominance, doesn't mean that leads to a safe world now, to our earlier point in China, the Cyberspace administration of China, the CAC is the largest regulatory body in China. And that's because it has a lot of responsibility and it's the key to the explosion of their 21st century economy and getting it right. We should think and have expertise. I want to first acknowledge all the libertarians who most Regulation has probably made a lot of things worse. I agree with that critique that there's a lot of regulation that's made things worse, worse. So I'm not for regulation that gets this wrong in which we just stifle ourselves, we lose to China, and then we even make it all worse and actually increase existential risk. And there's definitely a likelihood of that. But I think what we have to do is recognize that without guardrails, take your hand off the steering wheel. This will not end in a good place. I can tell you, for you and your children, this will not end in a good place. We need to do something we urgently need. I think you and I have talked about this actually in the case of social media, something like a Constitutional Convention or Bretton woods style thing, where you get the people, the technical experts, together with the legal experts, the people who work on AI safety and recognize what are the elements of coordination, where are all the shortcuts that we could each take in our companies, and how do we have all of us do integrated risk assessments with how the combinatorics of all these risks interrelate to make sure that we're all doing things at the same pace and rate to get it right. Importantly, China is actually not racing to just deploy large language model AIs into their whole society and having everybody use them. Because those who are concerned about the US China race, by the way, should really recognize this large language models are intrinsically unsafe for authoritarian regimes because you can't control them very well. So, for example, if you ask it about Tiananmen Square, what happened in 1989, or what is the significance of Winnie the Pooh? You can't protect it from saying these things you don't want them to say. And so they're being very conservative about rolling these things out there. And the US's paranoia is causing us to race to release it much, much faster. And by the way, I want to acknowledge in Steelman, all the benefits people are right now getting from that fast release cycle. There are many incredible benefits, but the question is, do we really need to do it at this speed, at this rate? If it means that we're just sowing the PFAS style externalities that we're going to discover 10 years from now, except they're not just going to be cancer in our bloodstream. It's going to be much, much worse and catastrophes that destabilize the function of society. So in terms of what things can we do, this is a really big question. And I think that what I really want to see is the experts who know the most about safety practices that we need asking, what would it take to get this right? What is the kinds of safety analysis? I'll give you an example. There's something in the community called evals. So you use an eval to figure out does this model, does, say, GPT4 have the ability to do dangerous things? Can it synthesize a bioweapon? Does it know how to make nuclear weapons? Does it know how to hack code? Can it exfiltrate its own code? Can it persuade a human or deceive a human? If I put it in a room with a human for an hour, could it get that human to give it $1,000? These are tests. These are dangerous capabilities evaluations, evals. Right now there are companies like Ark Evals that are testing models to know what kind of dangerous capabilities they have, but they're not testing comprehensively. They have a few things that they test for and it's all like, it's not enough. If it were the case that right now all the red alarm bells were going off for GPT5. So let's say GPT5 is being baked in the lab and we start testing it, and then the red alarm bells are going off ding, ding, ding, ding, ding on all the red alarms. You would think that they should hit the stop button and of course that's what they should do. And all the rest of the industry should also be forced to stop. But even if they were currently ringing red, there's no plan for how to hit the stop button. And let's say Microsoft, which is funding all of OpenAI's deployment, really, let's say Microsoft just spent a billion dollars on the marketing video for the launch of GPT5. They've already announced the press conference. They've already announced Microsoft Office 2025 that now runs your entire company for you with this AI that's doing all the automated decision making. Do you think that after they've made those investments, that even if the red alarm bells are going off, knowing that the other competitors are right behind their tails, that they're going to hit the stop button?

Right? No.

If it's Abstract.

But then the structure of regulation, that makes sense here. And it's hard to recommend this because existing regulation structures are so bad. But the structure of regulation is really the FDA structure. Right. So early on drugs went on the market and they caused harm. And then you sued people when they had drugs that caused harm. And then at a certain point people were like, this just makes sense. No sense. We don't have time for, you know, kill thousands of people before we're allowed to stop the deployment of the drug. So now we're going to say you got to pass the drug through a regulation and establish its safety and effectiveness before you're allowed to release it. You could imagine a similar structure here, that any deployment of something of a certain size has to be passed through a regulatory agency that runs the evals on it and it has to pass a certain percentage of the evals to be allowed to be deployed. Is that the structure that would slow this down in the way that you think it needs to be slowed?

Absolutely. I think that's the kind of thing now I hesitate notice we didn't start with that because if we start with that, people are going to think that's so draconian. You're going to get this slow FDA like process. Obviously. Is it possible to do that and get it wrong and to be over regulating a little bit? Yes, it is possible. But would you prefer to overregulate a little bit and go a little bit slower and still win the way race with China because we didn't shoot ourselves in the foot and basically destabilize our society in the process? And I think it's important for those who are really worried about the US China race, the social media companies said the same thing to the US Government. After all the harm said Mark Zuckerberg would say don't regulate Facebook. The US Tech companies are the crown jewels of our economy and our dominance. And if you regulate us, we're going to lose the race with China. I would argue it is the unregulated deployment of social media that has now caused the west to, to want to be. I mean, our kids want to be social media influencers and their kids want to be astronauts. That kind of tells you everything you need to know. And I think we have to care about the race and the global competition with China between the west and the East. I don't even like to use that frame, but I know a lot of people, if we want to make sure we're handling that set of arguments, we have to have a more comprehensive view of what our tech plus Society stack looks like, our Texas plus Society stack should be producing more, more cohesion, more trust, more education, more belonging, more community, more subjective well being, mental health, et cetera, collectively than the authoritarian AI plus Chinese tech societal stack. And that's the competition is the whole society plus tech plus culture stack in the west has to amount to something that outcompetes the authoritarian AI tech plus Society stack. That's how we should be thinking, Right?

That's a brilliant way to frame it. Because when you say we're going to lose the technology war with China, that's imagining there's one layer of society that is the determinant, most important layer. And what you're saying is actually when you add all the layers together, which I think most people would intuitively agree, that's what we should be worrying about. In fact, not thinking about the way they all interact in the context of social media has produced, introduced a defeat vis a vis China. China's in a better position with social media than we are today. And their kids are in a better position than we are today because you failed to think of it exactly this way. Now, I mean, it's one thing to think in the abstract about, you know, China versus the United States, but in the United States, the way things don't happen is that lobbyists show up on Capitol Hill and exert the influence of lobbying to stop things from happening. So in the context of AI, we know what's happened in the context of social media, but in the context of AI, where are the lobbyists and what are they going to be saying and what are they saying right now that might be resisting the push that you've been making to think about this in a more holistic way.

Yeah, it's really tricky. I think the lobbyist's job literally is to make sure that companies, the companies that they work for, have as many degrees of freedom as they possibly can to continue to operate in the world with as big of a moat over their competitors as they possibly can. That is just the role of their lobbyists, cynically. Right. If we just look at it from a pure power perspective, you are not hired as a lobbyist to constrain the actions of your client in its competitive race. So yes, is OpenAI, for example, and I don't want to pick on specific companies because it really isn't about them, it's about the broader race. But are companies like OpenAI lobbying for things like licensing, Frontier, what are called Frontier AI models, these big, the nuclear level, GPT4 level GPT5 level AI development. Are they pushing for having a licensing regime so you can only build those things if you're doing it with a license? Yes, they are pushing for that. Are they pushing for international coordination, something like an International Atomic Energy Agency for AI, where there's sort of a, an international auditing regime for where GPUs, which is the GPUs are the chips that go into building these AI systems. Chips are to AI what uranium was for nuclear development. So if we can track the flow of chips in the world and who has these big clusters of chips, that's what we need to know about. If we can track that, we can have some international coordination and monitoring of how that goes. But that also means you have to deal with black projects and governments that are buying chips in secret or those kinds of things. Is OpenAI arguing for or lobbying for the creation of something like an International Atomic Energy Agency for AI? Yes, they are. These two things are good steps. They also do serve the interests of the frontier companies because then there's probably going to be a small set of these actors that are licensed rather than many actors. Now they will point out that it is easier for the government to deal with say five AGI companies than if we, let's say say, succeed in the pause letter. We pause for six months and the top AGI labs pause. Now there's like 100 follow on companies that are now in that new race and it might be harder to regulate. So they're not wrong about that. It is a tricky competitive dynamic and there's multiple arms races happening simultaneously between corporations and also between countries between the US and China, between the decentralized open source world and the main centralized efforts. But really we're seeing some alignment with the major labs, AGI labs that are advocating for legislation that will create licensing, auditing, tracking of GPUs, some safety kind of risk requirements. But I'll give you an example, and you may know this better than I do. I believe it was the case that OpenAI lobbied against the EU AI act classifying its general purpose AI GPT4 as a high risk system, which would leave it vulnerable to the most stringent controls. And if they really were operating in the best public interest, they would want to adjust the regulation. So we had a risk mitigation framework that did limit it as a high risk system because it is a high risk system. So it's going to be confusing. Sam Altman's going to say certain things publicly that are going to sound really nice, but then we don't hear the conversations that happen behind closed doors with Congress members where their lobbyists end up canceling, canceling a lot of the things that Sam might have just advocated for in the public hearing.

Let's pick a particular context where a lot of us have been very anxious about the immediate term. So politics 2024, make tangible exactly how this AI could be deployed in a way that would be destructive, even more destructive than the AI that drove 2020.

So, first of all, because we actually haven't defined it yet, what is the risk? What is the space of risk? How do we define the phase space of possible risks from generative AI? So what do we mean by generative AI? It means that the ability to generate and decode language or media. I can decode a video. I can also synthesize a video. I can decode an image and know what's going on in the image. I can also synthesize an image. I can decode a movement of a robot. I can also synthesize the movement of a robot. The key insight, and I really do recommend everybody go back and see this talk we gave called the AI Dilemma, that kind of collects all of our views. One of the big insights is that starting in 2017, this new paper called Attention is All youl need birthed this new AI technology that's underneath the engine, underneath the hood of what we call AI. And that's why, for those who are wondering, like, why did AI feel like it started exploding only in the last couple years, like the progress has been explosive in two years? And the answer is because, just like the Indiana Jones swap at the beginning of Raiders of the Lost Ark, where you kind of replace the thing underneath the hood, what we replaced was the engine underneath AI with a large language model, a transformer. And the insight was to treat everything as a language. So the articulation of movement of a robot is a kind of language, the space of images. And you kind of push images in, in this big, huge, multidimensional kind of space in which there's images that Larry plus smiling equals. I can get Larry Lessig photos, and then I can say plus smile. And then plus smile means move the image space in the direction of Larry smiling. And it can kind of synthesize across billions and billions of really trillions of images and kind of decode what it means to move in this language space of ideas, thoughts, images, text, video, robotics, even FMRI scans, we cover all these examples. Okay, so now that we've sort of said, what is a large language model what can you do with that? Well, our society runs on language. Law is language. So I could say GPT4, find me a loophole in this country's legal system where I could literally get away with murder. And it's read the entire Internet and it's read the whole legal system of some obscure country and it could know a loophole in a law, in a contract. Generate media at scale to lobby Congress members who are against banning TikTok at scale. Now generate synthesized phone calls at scale to all the people that are their constituencies and text them and personalize it. And these are the kinds of things you can start to do. You can do some of these things before we've seen the personalized messaging AB testing on social media. But this is just supercharging all those harms. And I should say that the baseline space of harms, we're talking about generative media. You take everything that's wrong with social media addiction, fake information misinformation, polarizing content division, sexualization of young girls, bullying, et cetera, and you can just supercharge it with generative AI. So to baseline, what we're going to see is a supercharging of all the social media arms. That's on the bottom end. On the top end, what you have is when you can decode language at scale. Well, what else runs on language? Well, code is language. GPT4 find me a cybersecurity vulnerability in this piece of code now to give people a sense of why. Also, the language models are so dangerous. GPT3 was not able to find cybersecurity vulnerabilities and code very well. But GPT4 was all you did. You didn't change the entire base of AI that powered GPT4 to GPT3. All they did was just pump it full of even more information and refine the techniques a little bit. And there's these things called scaling laws. So the people who are really worried about AI safety now are worried because there's scaling laws that say when I just increase the number of parameters or compute or training, I can get jumps in capabilities that no one even intended anticipated. So, for example, we gave this example in our AI dilemma talk. I can't remember the dates here. I'm going to get this wrong, but theory of mind is a capability. Can I model what you're thinking right now? You're nodding your head at me for the people who can't see this interview. So I can read your mind and know that you're agreeing with maybe something that I'm saying they asked GPT3 versus GPT4 to read a transcript of what's happening between two people and then ask questions about what would that person have been thinking inside of this interaction. And so they're measuring essentially what theory of mind capability is GPT3 capable of? And it was capable of, I think basically the theory of mind level of a 9 year old. So think about how strategic a 9 year old can reason with you and what kind of strategic thinking and communication a 9 year old can do. GPT4 jumped from a 9 year old to more than a healthy adult levels of theory of mind capability. So now going back to the CyberSecurity case, if GPT3 couldn't find cybersecurity vulnerabilities in code, but GPT GPT4 could, I could say here's an SSH tunneling piece of code and boom, it finds some vulnerability in it. Now what if GPT5, which is only going to be next 12 months from now, what if it's extremely good at finding zero day vulnerabilities in code? Democracy runs on language. Media is language, conversation is language. When you can hack language, you can synthesize. Write me a 10 page paper on why the MRNA vaccine isn't safe and citing real charts and real facts and real statistics of people who died. Now point out everybody who said the vaccine was safe who was funded by a pharmaceutical company. These are all facts. These are not manipulating. A fact checker won't catch this. But now what I'm doing is I'm not flooding the Zone with shit or falsehoods. I'm flooding the Zone with spun truths, cherry picking data, misframing it, misrepresenting it. And that's going to get past your moral philosophy of fact checking as the solution. Solution to a misinformation problem.

Okay, so let's imagine you're a campaign manager and you have no ethics at all and you have the full power of all of these technologies and you know the capacity or the potential of all of these technologies, social media technologies as well as this generative technology. How would you deploy it to maximize your chance of winning? Like what would the objectives be that these technologies would attempt to achieve?

It's honestly a really good question. I have not gamed this out perfectly with the new capabilities. One of the things we'd like to do is actually get a group together to do a specific threat model on given between GPT4 and GPT5 level capabilities with both text, images and video. What are the kinds of threats that we really think we're going to see and how do we mitigate them? But people can guess the basics. You find political tribes that responded to certain memes, and you develop media at scale, and you start flooding the Internet and Twitter and Facebook and other places with that media.

But does it depend on. Does it depend on me believing that it's a person that's sending me this media? Is it, like, personalizing, and I'm, like, pretending to be a human, the bot is pretending to be a human, trying to persuade me that Joe Biden is actually a Chinese spy or something like that? Or is it just multiply? Is it just creating so much noise that the noise effect is consistent with what I'm trying to achieve?

Yeah, I mean, oftentimes, again, the Russians and Chinese goal is often not to get you to vote for a candidate, which, in fact, it's almost never to get you to vote for anyone. It's just about dialing up inflammation and cultural division, which is very, very effective. And just the cacophony, like another example to switch domains a little bit. If I was the Chinese Communist Party running 10 TikTok going into the 2024 presidential election, how would I screw up the United States in ways that had nothing to do with. There's this thing called Project Texas, which is, you know, TikTok trying to say, look, we're storing the data for TikTok on US servers on US soil in Texas. So therefore it's safe. You can audit it. Okay, well, if I'm the Chinese Communist Party, what if I do. What if I ship a. Just like, they have beautification filters that can replace your face and audio in your voice with whatever face and voice you want to be speaking from. Have you seen the film Being John Malkovich? Like, you know, right. So like, suddenly everybody looks like being John Malkovich and sounds like Malkovich. Well, imagine it's being Trump and Biden. So now they ship a Biden and Trump filter. So now every American posting a TikTok video pointing the camera at themselves gets to immediately speak in the voice and likeness of either Biden or Trump. So now I'm the Chinese Communist Party. I'm not even giving people a means to sow disinformation. I'm just sort of turning your society into a cacophony of Biden and Trump saying crazy things that everybody can now do in their own decentralized way. And I could even give people more likes and more visibility and more amplification the more you use those filters versus other things. And Again, none of that would be against the law. It would pass a fact checker. It's not about fact checking. It's fun, it's amusement. It's not about where the data is stored. It's just ways that we have opened up the parameter space for bad actors to manipulate our democracy. And nowhere is it written. I mean, when I met with President Biden, one thing we did talk about is the United States is unique, and that is it is a nation that argued itself into existence based on an idea. We're not just an ethnic population. We are an idea. And ideas are influenced by media and conversation. And if we can argue ourselves into existence with print, it was already the case that we're sort of arguing ourselves out of existence with Twitter. But with generative media, nowhere is it written that democracy can sustain this. I think. I think we need to be very aggressive at preparing people and the population to be much more resilient to the kinds of things we're going to see, which is less about these specific fake news stories and more about division itself is the thing that we should be cautious about. And I think that actually most Americans can agree that division is actually a unifying issue. That's something that people don't want. And I think I'm proud to say that in the social dilemma and the social media issues, that's what we found that everyone agreed that something that is rewarding citizens for being inflammation entrepreneurs is something that is abhorrent to everybody, regardless if you're on the left or the right.

So final question on this. You're closest maybe anybody in the country to an understanding of what people are actually thinking about in response to this threat? People, meaning government officials. Is there any optimistic story to be told about their ability to intervene even in the 2024 election to avoid the kind of dynamic that you're talking about? Like why. Why might you think the outcome you're talking about is not one we should fear for 2024?

You're saying that the bad outcome is not what we should fear.

I mean, is anything happening, Are there any regulations or any likely regulations that could step in, survive the first First Amendment challenge, and actually change the incentives to be this kind of inflammation engine?

So the reason for those who've been depressed by the last 40 minutes of our conversation, the reason that I focus so intently on the risks and the harms, is to clarify them, because so many people don't want to think or think that it's being overblown. So if we don't clearly communicate the Space of risks. We won't be able to create the immune response that we need to get ahead of them. The whole premise of all the risks that we've been walking through the last 30 minutes is to cause there to be the necessary changes that we need. That social media platforms should have a proof of humanity for each user with high accountability standards. That if you post something that goes viral that was fake or whatever, that there are bigger consequences for that, that we each maintain a stronger sense of care about what we're spreading into the information commons. I will not say that things are trending in a good direction on their own. That suddenly everything is going to. We're going to pass all the laws and we're going to change the engagement economy incentives and we're going to put in the guardrails for AI. The reason that I speak so forcefully about the risks is because I want that to motivate the kind of powerful response that we need. I do think that if we really got our act together, there are a bunch of big things that we could do to protect our society. In fact, I think it's an inspirational challenge because really the idea of an open society is presented with this and confronted by this new technology of generative AI and open society, I think have to be able to upgrade our previous sort of ideas about free speech and all the ideas that we hold dear for a 21st century technology Age which forces us to revivify what do we mean by these concepts in the context of factoring 21st century technologies. And nowhere is it written that that happens on its own. But the inspiring project is what is a unity of these two things? And I will say that Audrey Tang, the digital minister of Taiwan, I think is the leading the charge, as she has been now for several years, of actually conceptualizing and actually implementing in code and in tech and in societal laws what a 21st century digitally empowered democracy can look like. And for example, she's using generative media to more quickly identify areas of unlikely consensus between different political tribes. So maybe it was really hard to figure out, well, what would these different political tribes even agree on? But what if you could use general generative AI to actually find that common fault line, that common agreement line faster? Because we're actually using AI to do that. I think we need to actually think about how do you use and fully employ generative AI to strengthen the pillars of democracy, tolerance, trust, belonging, community and free speech and debating of ideas. But we have to not just treat it as that person arguing or ad homineming the other person is the kind of speech that we would be indifferent if that made up our information. Comments I think we should be rewarding the kind of communication that synthesizes across multiple perspectives. And the more we use AI to identify and reflect the complexity that is requisite of us to think about these complex problems that are facing us, we can be also collectively educating the population to think in terms of that complexity, doing these trade offs seal manning each other's ideas and the better we are capable of doing that, the more trust we can build with each other. So I think that that's the kind of inspirational kind of call to action that I want to invite everyone into is we need to reboot what a 21st century democracy looks like and not look to the past and assume the 18th century ideals are always going to work in the context of 21st century tech. And I think that's the project we're all on now.

So that's optimistic. And you've been enormously successful in helping people understand the threat. And that itself is, is an incredible contribution. And the challenge for us now is to take the threat that you've identified and figure out the strategy to get our regulators, our government to actually step up and and address that threat. Because you and I share the view that this is existential, not in the sci fi sense. I mean we want to get to that sci fi sense. We will be long gone before the sci fi sense kicks in.

And just to underline what you said, I want people to notice that all the risks that we've talked about in this last time we've been spending together did not involve a scenario where AGI woke up and AI ran off and took over our military and nuked us and nuked humanity. It doesn't involve any sci fi to know that there are enormous risks that derive directly out of the competitive pressures to race at the speed of deployment and recklessness rather than race at end the the speed of getting it right and we can get it right. But I think you know what I would have preferred rather than a six month pause. I think that was misnamed because six month pause makes people think, okay, sit there and twiddle your thumbs for six months and don't do anything. What it was really about was a six month redirection of just like Bretton woods was six months, right? Six, six months at the Mount Washington Hotel in Vermont or New Hampshire, excuse me, where you know, the hundreds of delegates got together and actually said how do we create a post nuclear world economic order which has Lower rivalry between nations because we have more of a shared economic system and more vested interests between each other. We are now at the post AI age and we need something like another six months in a hotel where we think through really deeply how do we get this right? And that's the kind of continuity of effort. I mean, literally, I've actually been thinking about a retreat center. You get the hundreds of people together to have not social signaling and kind of boost jousting of their intellectual ideas, but the actual implementation of what is the North Star we want to aim for. How do we reboot these 21st century ideas of democracy? And then let's build it, let's create the action plan, let's coordinate. And if we spent six months on that and imagine all the smart people in Silicon Valley were on board, instead of vilifying them, they're on board with this because they live in the democracy that we're talking about undermining. And so they need this too. So that's the inspiring vision that I would like to see happen. And you know, it's not out of the question.

It's the project we have to do. I'm so grateful for your work and I'm thankful you've taken some time to help explain it in the context of this series. Thank you, Tristan.

Thanks so much, Larry. Always been inspired by you too.

That was Tristan Harris in this preview to the next season of Another Way. As I reflect on that conversation, which I've had many with Tristan, that was for me one of the most enlightening I saw for the first time. What really terrifies me about the business model or the business dynamics that we're seeing emerge in the context of AI because as you heard in the conversation, it's perfectly understandable, understandable why there is this race right now among these very big and powerful players and just this week, including a new company by Elon Musk, to gain as much market share as they can so that people build their next generation development on top of the platforms that each of these dominant companies. Companies is deploying. And the danger to that, of course, is that these are not safe platforms. Now, safe means what a lot of people think that the fear we should have about AI is that they're going to eventually launch nuclear weapons against us humans because they've decided we're bad for the planet, or that they'll overrun humanity with some overriding purpose that they can't figure to mitigate the harms from. So we suffer as a consequence, the whole range of sci fi threats but as I'm going to emphasize in the book, which this interview is taken from, from the real threat that we face is not a threat from technology. The real threat we face is our incapacity to govern in the context of dangerous technologies. My own view is that AI has enormous potential to do extraordinary good, and at the same time it has enormous potential to do extraordinary harm. And the ideology that we have that we should just step back and let things develop has never been more dangerous than now. No one thought we should step back and let nuclear weapons develop in the hands of private parties developing their weapons as they saw fit. For some reason, it was obvious that that dangerous technology couldn't be be allowed in the world uncontrolled. And while I don't think computers and AI are the equivalent of nuclear weapons, I do think the need to govern to avoid the catastrophic consequences of misuse is just as pronounced. Do we have that capacity? Do we have the ability, when we look around at the news, covering every irrelevant issue as loudly and as dramatically as they can, recognizing all of the extraordinarily serious issues that barely get a mention, can we really govern this? That's the question of the series. And this is the introduction, the preview of that series. Next week I will release the second of the two interviews we will do in advance. That interview is with Ron Fine of Free Speech for People about the legal argument against super PACs. Stay tuned for that second preview. Thanks so much for listening These podcasts. This series are produced by Equal Citizens. You can find us on the web at Equal Citizens Us. And if you have ideas or questions, you can find a page on that website where you can raise those questions directly to me and I'm happy to answer them. Thanks so much for listening. Please share and subscribe to this series, which will begin in September in the arc of the book. But next week we'll have the second of the preview interviews released. Thanks very much, Sam.