Ep 102 - Alex 'Sandy' Pentland on Humanizing Technology

Get ready to enter a brave new world with your host, Vasant Dhar.

For more color on the podcast and additional commentary, please subscribe to my newsletter at vasanthdhar.substack.com. Hello, and welcome to Brave New World. My guest today is Alex Pentland, better known as Sandy Pentland. Sandy is on the faculty at Stanford University and MIT. He works with governments, corporations, and not-for-profits to deploy systems that touch billions of people's lives and aim to complement human nature rather than treating people as cogs in a bureaucratic machine. His recent book, Shared Wisdom: Cultural Evolution in the Age of AI, talks about how to implement AI in a human-centric way. Sandy, welcome to Brave New World. Delighted to have you on the show.

Happy to be here. I've known you and your work for a long time, and so it's great to finally be part of this.

Yeah, likewise. I've actually followed your work for a long time myself, and so particularly happy to be talking to you. For the audience, Sandy, say a little bit about yourself. What influenced you growing up and how you ended up at the MIT Media Lab? What's your journey been like?

Well, it's sort of crazy, like many people's lives, right? I started off in Ann Arbor, Michigan, and originally thought I wanted to be a psychologist, and then I learned that they didn't know enough about people to attract my attention. Ended up doing my— I took a job as a programmer because I didn't know what else to do. And it was in this place that worked with the very first land use satellites. So like we're doing things like counting the number of beavers in Canada from outer space. Well, that's sort of cool. But I also learned that the engineers don't know anything about, like, for instance, human vision. Humans could do things and the computers couldn't. And so I ended up going to MIT. As a student and then later as a postdoc at Stanford and SRI, the Stanford Research Institute. And then when the Media Lab started, I said, well, this is great. Okay, so I joined there and became department head and suffered the sort of things that administration and universities have to suffer. And then more recently came out here to Stanford because this is where AI is happening. And it's really interesting to sort of have your finger on the pulse and See what Kool-Aid they're drinking, because everybody here has these sort of attitudes about things that maybe aren't shared in the rest of the world.

So what Kool-Aid are they drinking out there these days? I'm actually coming out there next month myself. I teach this tech MBA class and we visit the West Coast. So I get a yearly dose of the Kool-Aid. So tell us what's going on there these days.

The big thing, of course, for the For the last little while has been this frontier models, edge models that are bigger than anything else, and the hubris that we're ahead of everybody else. I was at a meeting about a year ago where people were saying, well, we're about 10 years ahead of the Chinese, and so we have to be able to think about how we can use that as a strategic advantage. And I was like, no, boys and girls, you're not. You may think you are, but you're not. And then I was at a dinner with the head of Singtel and the head of the Singapore Investment Fund. And, you know, we heard a talk by the VCs out here about, you know, what they were doing. And the Singapore guys were just smiling because that's not the game that everybody else is playing. Frontier models? Yeah, good. Spend your money, right? But what we're trying to do is actually put little AI things everywhere in the whole world. We don't want a big centralized type of a thing. We want everybody to be running their sort of our software, but for accounting and for payments and everything everywhere in the world. And so everybody else is playing a different game than the US, and I think their game is actually a lot better. And then excuse me for going on here, but, and now everybody has sort of Focused in on the fact, oh my God, these big, you know, edge frontier models are too expensive to make money at, right? Too many data centers. So let's have small things that do small jobs like accounting or whatever really, really well. And those turn out to be cheap and you can make money at 'em and they don't hallucinate as often and blah, blah, blah. Everybody, and I mean everybody, is building these AI agents. And that's the name of the game now, AI agents for everything.

And it's interesting because you've been into sort of distributed computing and agents and stuff for a long time, right? I mean, I recall you. Absolutely. Yeah. And so what does that feel like to you? You know, that it's sort of come full circle in a way?

Well, it's great to see. A little sort of brag is I gave a keynote at JP Morgan and they introduced me as somebody who was 8 years ahead of reality. Well, you know, as an academic, that's pretty good. If you're starting a business, that's really bad. But, you know, it felt good in that sense, right? So yes. And the thing that people don't realize now is that the things you read about are moving that way quickly, but the finance guys are already there. They've got blockchains, they're doing AI on the blockchains. You know, they're already beginning to move large amounts of money. This is not crypto. This is SWIFT and American Express and Visa and things like that that are really changing the way money moves around the world using these AI agents. And it's a little scary because we don't really know how reliable they are, and particularly when you get networks of them, the evidence is that there's problems and we don't really know how to A, find the problems and B, fix them. And so it's a little interesting to see people jumping in this with both feet and they haven't quite tested the water yet.

Yeah, yeah, no kidding. I want to get to your book, which is Shared Wisdom, but this is really an interesting sort of prelude to that. And as you said, there's so much going on, the Kool-Aid and all that. And these days there's talk of the AI bubble. I mean, people ask me, do we have a bubble? What's your sense of this? Is this a bubble? And if so, in what sense?

Well, yeah, people talk about this, but I think, look, when we built the railroads in the US, That was a bubble, but we ended up with railways and we ended up with some very rich people. A lot of people went broke. That's gonna happen. Happens every time you get a wave. The thing that changes it here is 2 things. One is everybody in the world wants AI. If you go to other developing countries, they say, look, we're not gonna be making money. Our export-import, our trade, our manufacturing needs to be as efficient as the rest of the world, which means we need to have these little AI agents everywhere. And so there's really this sort of insatiable demand from, you know, 6 billion people to have these sorts of things available. And you can move the AI anywhere in the world. So the fact that they're building, you know, centers here and not over there, you know, in a way doesn't matter. What does matter is that they be in the local language and they deal with the local problems and things like that. And so you begin to see that really taking off. And that's a huge demand. That's like, it's like the mobile phone industry. It's, it's incredibly big. Plus, you know, the way ChatGPT has suddenly begun making money is the way WeChat makes money in, in China, which is not through advertising. I mean, they do that, but it's through helping people buy things. So you can now buy things from Walmart Through ChatGPT, you say, "Buy me a mop," and it will go buy you a mop. Literally buy you a mop. Mop shows up at your door. So that is a very profitable business. All I mean is a little bit like MasterCard or Visa or something. If you just take a little tiny slice of that purchase, you make huge amounts of money. And you see this in China where WeChat has this sister organization called Ant Financial, and Ant Financial. Takes a little bit of slice of everything people do on WeChat, and boy, are they rich.

Yeah. I mean, it's amazing. Things are just moving so quickly. And as you said, does it really even matter where these data centers are? Which makes me wonder, what does it actually even mean to be ahead? What does that even mean? So when people say, well, we're 10 years ahead of China or 2 years or whatever, what does that even mean?

I don't think it means all that much because, you know, what's going to end up is a world where you have these little bits of AI everywhere. They're calling them agents. Good. And if they're everywhere, then it's a business that's a little bit more like Microsoft. It's like, yeah, we know we sell the little pieces that run the world and— or Oracle or SAP or any of those sort of— it's just part of the infrastructure and it's everywhere. And so the localization is not, except for energy and cost of energy. That's like I was told recently that the largest solar panel array in the world is in the Himalayas in Tibet because the sunshine is more energetic there. The photons have more energy so that the solar cells work better. And guess what all those solar cells are for? Data center. Oh, okay. So that's going to be a cheap place.

Are we going to get like a data center in Tibet?

Yes, we have one. That's amazing, right?

I was not aware of that.

Because the energy is less, right? Less costly and hydroelectric. I mean, where are you going to put up your thing? Well, someplace where it's not just green, but cheap. So Yeah.

And it comes down to like, you know, what, you know, it's like unit of intelligence per unit of energy, right? Like what, and that's what it comes down to.

That's right. And recently there was a paper in Nature, which I'm sure you've seen, Nature Machine Intelligence, showing that the, for a given level of intelligence. So like, think about, I'm sorry to hit on accountants all the time, but that's my Job of the day. For a given level of intelligence in accounting, the cost of that is halving, going down by 50% every 3.5 months. So yeah, that makes Moore's Law look like nothing. Moore's Law has been the thing that's just shaped our society incredibly in the last 20, 30 years. This is Easily 5, 6 times faster. And that's why you see so much disequilibrium is that, yeah, stuff that used to be impossible is now cheap. And that will continue to happen. Like, for instance, a friend of mine who manages a large sort of AI projects said, here's a choice. So you could have a certain amount of money, you could start a project now at You know, spending X dollars per month and have an 8-month project, or you could wait 4 months, spend 2X, and only have a 4-month project. Which would you pick? And the answer is you wait 4 months because 4 months from now, the tools will be twice as good as they are now. So that, that waiting 4 months buys you this huge thing. But of course.

You know, it's funny you're saying this because, you know, one of the, you know, I was giving a talk at a large insurance company the other day and they were saying, you know, what are the implications for senior managers, right? Because there's so much FOMO around AI, right? And especially that MIT study that showed, you know, 95% of the projects didn't deliver.

Which I don't believe is really right.

Yeah. You know, what that made me really wonder was like, What projects did these people know what they were doing? But coming back to the question about how should we think about it, and my answer was think slow, that there's no hurry here. This thing is not running away from you, right? Make sure that you think through what exactly you want to do with AI and the kinds of ways you expect to transform the business, but don't rush into it, right?

Well, this is what my Shared Wisdom book is all about.

Yes, which is what I want to get into.

Yeah. Like everybody says, oh, you know, we're all going to be dead and, you know, the robot overlords or they're, you know, we're going to be lotus eaters and it's all going to be, well, no, it's not going to be any of those things. It's going to be something in the middle. And we ought to think about what we want now so that we get there. And so that's what the book is about. Yeah.

Yeah. So, so let's talk about that book, Shared Wisdom, you know, and, you know, interestingly, you know, my, my last podcast guest was Deepak Chopra, and he's said a lot about wisdom. And it was interesting, he's written these 2 books, one called Digital Dharma and another one called Awakening. And I read Digital Dharma, and he's talking about how the wisdom of the ages is now available on your fingertips. And I hadn't actually even thought about it that way, that actually wisdom is one of those things that you can actually ask these language models about, tell me about the Vedanta, or tell me about Taoism and stuff, and it gives you pretty good answers. Oh, yeah. It's not hallucinating. It's actually giving you really good answers, and it becomes like a source of learning. And I hadn't actually thought about that before I read his book and had him on my podcast, but you talk about shared wisdom. So, Tell us what you mean by wisdom and how we really should be thinking about AI. And in your book, there's several themes. You talk about stories, community, consensus, wisdom. So talk about those things and the sort of key themes and messages in the book.

Well, I think the sort of top-level thing is you want AI that fits people. And we've tried AI a couple of times, expert systems, logic systems, machine learning, and it just hasn't sort of taken into account the way people work. So getting people to do things as a group, so collaborative action, right? It's what we try to do for organizations, for government, for everything, requires everybody sharing a lot of not just knowledge, but what I am calling wisdom and what Deepak Chopra calls wisdom too, which is, uh, and the Pope. The Pope wrote a book called Sharing Wisdom recently. You might know about that. Um, but what it really is, is the things that people agree are effective, right? Are true and effective. So like, if you do this, then this is likely to happen. When you have something like that, uh, that everybody sort of knows, then you can convince people to act. Right? If people don't agree what you should do and what the outcome will come, it's very hard to get them to act. You have to force them and it doesn't really work and so forth. So what an organization is about, a company for instance, is about taking all of the local things that you do and combining those into have a company culture. So that's the company's wisdom. Doesn't mean it's correct. Doesn't mean it's true. It just means it's what everybody believes is the right way to do things. And that wisdom, that culture is what constrains what people do. And it says, oh, well, you know, at IBM we would do things this way. That's the IBM wisdom, which may or may not be effective at any one time in history, but it's what we do as a society too. So what the book says is, look, let's look at how that Collective action is enabled? How can we get people to do things that they want to do? I'm not telling people what to do. How can we organize ourselves best? And how can we use AI to do that? And there's a couple of things you can do. One is you can find out what everybody else is doing. So that's social networking, but trying to avoid the echo chambers and things like that. A second thing is getting people to talk to each other, to be able to have a discussion about what everybody believes, wants, and get them to understand each other. And we have all these sort of things about polarization in our society at the moment, and, and there are actually fairly effective ways to get rid of that, and they've actually been deployed in various ways. Like, for instance, in Taiwan, they use this type of very simple AI, um, the system called POLIS. It's, it's Sort of an older system now, but it's good. They went from 7% of the citizens trusting the government to 70% of the citizens trusting the government in a period of 10 years.

So say more about it.

So how do you actually restore trust in government? And the answer is fairly simple, is you give people voice. You make sure that they're heard. And so for instance, we created a system which is being used in Washington, DC that takes the best learnings from that, which is that No loud voices, no advertising, no influencers. You don't get, no fights. So what you do is you get to contribute comments to a common. A second thing is you have tools for finding out so any person can see what everybody believes, right? 'Cause so much of this echo chamber is that you only hear the loud guys and you don't hear what everybody else believes. So it turns out, like for instance, in this country, the US, that People actually, by and large, agree about issues like abortion and gun control. They're really pretty much on the same page except for the loud voices on the left and the right. And if you see that in a graphical form, people go, oh, actually, I believe like everybody else believes. I'm gonna stop being so polarized. And it actually works. So, deliberation.io. Open source, you can steal the code, whatever you want. And Washington, D.C. is using this to try and help them understand what the citizens want out of government. You know what the number one thing that the citizens wanted out of government was?

What?

A personal agent to help them deal with government, because government is too complicated, too many rules, too many forms. They wanted an AI that would help them defend themselves essentially against government and take advantage of the things that they were supposed to have. I find that really amusing. So, so people aren't really afraid of AI. They're afraid of other people using AI on them.

One of the things I actually muse about in my book is whether AI can actually make government more transparent, right? You talked about an agent to help people deal with government. And part of that is because, you know, government's become so opaque, right? No one has any idea what You know, actually goes on. You know, it's become so complex, it's become so opaque. So can AI actually help make government more transparent? You know, I was on a podcast yesterday and someone asked me this and I said, yeah, I believe so because it can help us with sensemaking. As in analyze the B-52 bomber, the Trident submarine, and the Minuteman missile projects and tell us about their snags that might apply to the current F-35 and Polaris projects.

Well, part of this is the more general question of how do we know what decisions are being made about us from the AI? Is the AI doing good things too? That's a very closely related thing. And this is a project that I've been on for a long time. I helped the UN Secretary-General do exactly those sort of things for the Sustainable Development Goal, which is to say, how can we have sort of an audit trail of what's happening, we can then examine using AI tools to find out what's really happening, right? And how can we make that open so that everybody can see it from their perspective and have a fact-based discussion about it? And you're right, government's not transparent by design because they don't want it to be transparent. And as a consequence, it's not very accountable. The same is true of, you know, organizations and all sorts of places where— courts, for instance, where, you know, we'd like to have a little more transparency and accountability. One of the big problems is there's so much data that could be there that isn't there. And the other is, if you had all that data, how would you make sense of it? Right? And the goal there is to come to this sort of collective wisdom, the shared wisdom, which is if enough people can sort of see what's going on and talk about it, then they can come to a conception about what's right and what's wrong and what actions to take. And that's the whole point of the shared wisdom, is you have to get people to be able to understand what's happening and agree on a good path forward. And AI can definitely help that because it's able to look at lots of data and provide things. It has problems. It shouldn't be making the decisions. It should be helping people make the decisions. Let me give you an example that I think is very illustrative. I was on a panel with CTOs of 5 of the largest corporations in the world, and they were talking about the difficulty of deploying AI and getting the data and stuff. But they also all mentioned that they had given every employee an AI buddy. What is an AI buddy? Well, it's a little local AI that has actually read the manual, has actually read the newsletter, actually knows what's going on in the organization, which allows employees to be much more in the loop, right? Because nobody reads the manual, nobody reads the newsletter. So, what this little local AI that's for you, right? It's local and knows what's going on. It can tell you, oh, so-and-so down the hall is doing that, or the people in this other country are doing that, or You know, that's not the way people usually do it, or in this situation, this is the way people do it. It's sort of a repository of the company's wisdom or the company's culture. And I talked to the head of a large— one of the largest consulting companies. I spent almost half a day with the chairman of the board. And what they were saying was, look, if we can actually make people more in the loop, more aware of how the company is thinking about things, the shared wisdom, then we can have things that are much more distributed. We don't have to have everybody come to the one central office to be able to talk around the water cooler anymore. We can have people be plugged in even though they're not physically in the same space. And I think that's a pretty good use of this type of stuff.

And interestingly, that's a change that COVID kind of engendered as well, right? This ability to be distributed and work more flexibly. I mean, one of the more desirable aspects is that we can do this work in a more distributed kind of way. And as you say, if you have the knowledge, then it's possible. You don't need to get people in the same place.

Yeah. I mean, that was sort of One of the things that I certainly was brought home to me, like everybody else, is I don't have to be in Cambridge, Massachusetts to do the things that I do. I could be anywhere in the world, but I have to talk to people regularly using things like this. And we do have to get together from time to time. And the reason for that is that not all the semantics, all the meaning is in the words. It's also in how you say it and your attitude towards people. And that actually requires— a little sort of like getting together still. Someday it may not. Someday we may live, have really adequate VR, but I think that's actually a lot farther off than at least some people would have you believe.

You know, one of the things that your book made me think about, you know, because you talk about culture being transmitted via stories, you know, that these stories, you know, are enduring and they, you know, they carry wisdom and I was talking to my colleague Jonathan Haidt, who was an early guest on this podcast. Sure. And one of the things he said to me was that, or made me aware of, was that something like 90% of the content consumed by young people these days was generated in the last 2 days, something like that, as compared to, let's say, when the previous generations where we watched Bugs Bunny or Roadrunner or something, and the content was actually created by the previous generation. It was created in a certain way and we sort of consumed it, and there was a sort of a continuity. And I wondered whether humans are sort of losing this social and cultural memory faster than any time in history. Your book made me wonder whether that's actually happening.

I think what's happening is the motivation for my book, actually, is that Culture is changing faster than it ever has before. And, and, you know, we're sort of aware that culture didn't change much from, you know, the time of Christ to about 1500. And then we had the Enlightenment and things changed a lot, right? But nothing like today. And the reason is, is that in the Enlightenment, suddenly people could talk to each other via post, letters, things like that, right? In ways they never could before. And now we have these new ways of communicating and concentrating information. As you said, you know, these AIs are repository of culture. It may be that they are the thing that are our greatest anchor to culture, because the things that the kids are creating make reference or are in contrast to the historical culture. So they use the same words and concepts, but they use them in different ways, which is the way innovation happens. Another issue is, of course, They tend to do it online and not in person. But I think that at some point that will reach an equilibrium. For instance, Facebook destroyed a lot of culture, of neighborhoods, of things like that. Not just Facebook, but all those tools. On the other hand, today, if you look at Facebook groups, they're all, almost all of them are around physical meeting places. So it's the local basketball team, it's the local church group, it's the local government. It's things that people actually do together. And there's now this sort of digital side to it too. So the physical hasn't gone away in that case.

That's interesting. I was actually going to ask you that, whether physical has actually gone away. It made me wonder, what does community even mean now? It used to mean a neighborhood or something defined physically. And what does it actually mean now? It made me wonder. And it's interesting you're saying that it's now this sort of hybrid of a physical shared space, but also like a digital shared space, right? So the notion of community has kind of become this hybrid of physical and digital.

Yeah, I think that's exactly right. And one of the things that's a real problem with social media, with AI and things like that, is they don't have any recognition of the notion of community. What we do as people, and this is Danny Kahneman and behavioral economics and all that sort of stuff, what people do is largely a function of the culture of the communities, plural, that they're part of. So you have a work group, you have a home group, you have other sorts of things that you're part of, and those all have a collective wisdom about how we do things and what we do and what we value. And that's why you joined the group. is because you agreed with that. And what we don't have in our digital tools is very much recognition that of all. So you take Facebook, which is about having 1,000 friends, and people have manipulated into this thing that is actually reflecting real community. It's no longer physical community in the sense of the 4 blocks around you. It's the places you spend time, and we're much more mobile now, so it's all over the city region. Mm-hmm. Or it's a professional community, or it's a community of practice, which is physically distributed. And so I think one of the things that people repeatedly do is they try and take these digital tools and twist them to support friends and family, community, things like that. And I think there's a real opportunity for people who are inventing new technology for these new AI things to explicitly recognize community. Yeah. So, one of the projects that we're doing is called loyalagents.org, all one word, and it's with Consumer Reports. So, Consumer Reports is an example of a community that comes together. It's 8 million people. It's not a small thing. It's the whole country come together to help them make better decisions about purchasing and about safety and about services. So, what they do is they give a little bit of money to Consumer Reports, and in return, they hear what the community thinks about each project, each product, each service, scams, things like that. And so we're working with them to provide agents to all of the community members that will help them deal with the complexity of buying, selling, dealing with government, things like that. And so there is a possibility of having these new sorts of tools be very community-centric by having cooperation within communities supported by the AI agents so that they can, as a group, defend themselves against the big guys. Same sort of thing that happened, you know, when we set up agricultural cooperatives back in the late 1800s, when we set up Credit unions, labor unions, people, groups of people come together around these new tools, these new media, to be able to defend themselves against large corporations, large government, and so forth.

Yeah, so I guess it works for an outfit like Consumer Reports because you don't get this sort of massive influencer effect, right? It's just more moderated. I mean, it reminds me of Stack Overflow or groups like that where there's some sort of agreement on what sort of a good solution was or a good question was, and there isn't this sort of heavy influencer effect that's dominating the sort of, or creating groupthink or something like that.

Yeah, that's one of the very principal problems with the normal sort of social media people use, and it's why they're turning to things like Reddit and Stack Overflow and the groups in Facebook and stuff like that is because You can't do anything sensible when there's these loud voices around. It's just, it's like trying to have a political debate in a bar. It's a bad idea. Don't do it.

So, Sandy, I want to segue into sort of a related topic. And I, you know, I did warn you that I would ask you about this, which is this whole issue of the governance of AI. You know, one of the questions I raise in my book is, you know, will we govern AI or will AI govern us? Part of my thinking is that to a large extent, AI is already becoming a gatekeeper of human activity. You apply for a job, your CV is looked at by an AI. Increasingly now, people are being interviewed by an AI. So part of me feels like we've sort of slipped into this Huxleyan kind of world without realization. There's been this sort of gradual disempowerment. Yeah. And that we're sort of slipping into this, the world where AI becomes the gatekeeper and it begins to govern us. How do you think about this?

Well, it's really a key question. The first thing I'd point out is it's not new. When we had the version of AI that was expert systems, banks, hospitals, everything have these fixed rules for everybody, right? And we're just sort of used to that now. But really what it is, is these rules, these And they're implemented by computers. I know a person programmed them, but it's a computer thing optimized by computer. Now tells you whether you're creditworthy. Well, it doesn't know you. Different communities have different rules and different ways of doing it, and they just shatter that. And that's why we have this sort of destruction. One of the major factors why we have this destruction of community is that everything has become uniform. Everything has become centralized. It's these electronic systems, these sort of big bureaucracies that run things. Now with AI, you have more of that. Where there used to be people in there, there's now this AI chatbot thing interviewing you and so forth. I was part of the discussion for the EU AI Act. So this group that includes the prime ministers and the presidents of every country plus the regulators, and we were talking about it, and the straw man I put forward was what you need to have is you need to have transparency and accountability. And that's implemented by having audit trails so that you know what's happening and that's publicly available and strict liability. So when somebody does something that's not good, you can see it immediately and you can nail them. And that's the way we do things in lots of areas, in safety for cars and safety for electricity and safety for all sorts of things. We know what the harms are. We know what's bad, pretty much, right? Killing people's bad, discrimination bad, et cetera, et cetera. And what we don't have is a good way to connect the AI to those harms. So if AI was forced to put off audit trails the way we do with anything having to do with money, no corporation, no person in the world has things that they keep track of where the money came in and where the money went out. You have to do that for your taxes. You have to do that to make sure things aren't going wrong in some way. Audit trails are not scary, nor are they expensive. Everything can do it. What's not present today are the audit trails for AI. What came in? What did they decide to do? What was the decision? And How did that turn out? And if that were available in a sort of semi-public way where anybody could audit the company and the AI using the AI also to check it, right? Then you'd be able to discover discrimination and bias and self-dealing and things like that. And I think you're absolutely gonna need that because first of all, AI is moving into finance. So all those rules about self-dealing, about bias, about discrimination, redlining, all of those laws apply now to anything that's doing stuff with money, which is going to be most of AI. And we have to be able to audit it. And the part that's missing at the moment is strict liability, which is if your audit trail shows that you've been cheating, you're guilty, right? Today, you hire a whole bunch of lawyers, you spend millions of dollars defending yourself over data that doesn't— there's no data to really base the legal case on because there's no audit trail. So we need to have a lot more transparency and accountability. And I think that's the thing that will clean it up. Now, when I did that with the Europeans, all the presidents and prime ministers said, that sounds great. That's the way things work. We could just do it. But all the regulators wanted to have very specific rules. So they crafted all these rules and details and things and made this huge sort of bureaucratic mess, which they're already backing away from because it's just unworkable. The thing changes too fast. It's too much about litigation. I mean, it's the EU AI Act is the Lawyer Support Act. Another one. What you want is you want something where It's transparent enough that it's very clear when things are screwing up and who's responsible.

But what's your take on my question though about whether AI might end up governing us or whether we will govern it?

Well, that's the important question actually in many ways. So the whole idea of what I did in Shared Wisdom is say, the one thing that we don't want to do is take away agency, the ability to make decisions from people at any level.

And yet we're giving AI more agency, right? I mean, there's, there's all this excitement around agents because we give them agency. And that seems to be this sort of tension, right? Where we're giving them agency because they're so good at doing these things for us.

Well, I think, I think there's, there's lack of clarity about this. So for instance, let me give you another example. So a doctor is somebody you give agency to, but they have a legal duty to represent you and not them. Okay. So that's called a fiduciary duty. It's a duty of loyalty. It's why our project is called Loyal Agents. You want agents that represent you, not them, not the person that wrote it, Not the company that it's buying from. And there are legal standards for this already, because this is a problem with shyster lawyers and bad doctors and all sorts of things. So we have dealt with this for a century or more. We sort of know what bad looks like, but we haven't been applying that standard to AI. And to apply it to AI, you need to be able to have an audit trail. Surprise.

Yeah, but also, as you point out, like some sort of duty of care or a fiduciary responsibility, right? And like physicians have a duty of care, like mental health experts have duty of care. And yet we have, you know, these AI companions that seem to be proliferating without any such expectation of duty of care, right? And we've had these, you know, some tragic incidents where, you know, a teenager killed himself because he thought that the companion was actually encouraging him to do that, whereas he had no idea that the AI had no idea what it was really doing. So how do you think about that? Because at the moment, there seems to be no duty of care. And this is, when I talk about what does it mean to regulate AI, that's one of the questions I have is in what domains do we need to think about duty of care that's preemptive in terms of harm? Because one of the things I— picked up sort of at the end of your book, you talk about this sort of ex ante versus ex post kind of view of regulation that really sort of resonated with me as well because at the moment there is no credible threat of being punished for consequences of demonstrable harm. And so it's kind of a Wild West kind of world out there. And while I'm sensitive to the fact that Overregulation can hamper innovation. The question in my mind is, in which areas should we think about regulation in a preemptive way, like as in duty of care, and in which areas can we actually think about it in terms of, okay, there's a liability that there was clear harm that was demonstrated, and so that's sort of an ex post way of thinking about regulation. I found that sort of interesting.

So, so 2 things. One is you should be careful about using the words duty of care because it's different than duty of loyalty in law. And duty of care is actually harder to prove that somebody violated that than negligence. So you absolutely do not want duty of care. You want duty of loyalty because that's the one that's more centered on the human. Does it actually represent the human of the, uh, the interest? The duty of care says, nah, you weren't terribly, uh, negligent, and that you have to now prove that, uh, you had a right to expect them not to be negligent. So it's, it's a little technical thing, but it matters because the lawyers, uh, argue it that way. The big thing that I was just talking to somebody about this yesterday, um, The big thing that Silicon Valley has done for a long time is they do things that don't need to pay attention to regulation and to harms. Write software, you decide to buy the software, you are a platform like Section 230. It's not what they're saying, they're just helping move the bits. They've gotten used to this idea that they can ignore law. Okay, but when you come to something that is an agent, that has agency for you, there's a lot of law. Like doctors, like lawyers, like accountants, etc. And for the things that they're building, they have legal implications. Now, it hasn't been completely argued out, but part of our Loyal Agents project is involving the law school, right? Because this is This is where does law really bite these things and where it doesn't. And what people are beginning to understand, these companies, is they're now operating in an area that is a regulated area. When you take agency from people, it's a regulated thing that you're doing, just like moving money, just like medicine. And you have to begin thinking about that. And we haven't had the lawsuits, the civil suits, etc., yet because the lawyers are just catching on to this and people are just catching on to it. But there are already a lot of very serious conversations about it. There's a number of state bills in California that are actually, you know, been put forward to address this type of a thing. So I'm very hopeful that this notion that we have of fiduciary duty, duty of loyalty, That we have for doctors and lawyers and so forth, uh, will get clarified for agents so that, uh, you can, you know, make them sit up and do the right thing, right? Uh, it isn't gonna happen automatically. You need to actually sort of get out there and do it. But, um, I think the cards are in the hands of the citizens at the moment, and we need to play them right. But we have the cards.

So is the current legal structure adequate to deal with sort of the age of AI? I mean, can we use like tort laws, for example, as a way of enforcing whether there was negligence, or do we need new kinds of laws?

Well, I think that just when you buy something that's got electric cords on it, there's this little thing called UL tag on it. That's an organization, a not-for-profit organization that certifies that the design is good enough. And what we need is that type of certification for AI, which is gonna require a little bit of learning. And so I think that the existing regulation, if it is sharpened up just a little bit, which is what we're doing with the law school and the lawyers and et cetera, right? Is good as a start because we know a lot of the harms. We know how to connect the harms to the AI. We need to just make that happen. So the existing tort law is adequate to extend to that. It's gonna take a little bit of extension work to do that, but it's not a lot. When we do that, we will eventually find things like we found in social media, which is that there are long-term effects. That are difficult to measure and are not covered by tort law. So for instance, the sort of echo chamber type thing that you see in social media, some of the things you see about children's learning being affected, those will take a little while to show up. And there may be new sorts of harms, and those will require new regulation. But we need to actually have experience with this. We need to have a body of evidence. To be able to say exactly what's happening and why and where regulation will do good and not just cause more problems. So the big thing is starting with existing tort law and making sure that connection is done. And incidentally, it's not just us. We often discuss as if it was only like the US you needed to deal with. Well, Well, we have all this international trade, we travel internationally. All of the regulations need to be compatible across countries. Otherwise, you're going to be getting these AI agents that come from God knows where, that have God knows what sort of certification, and they're going to rip you off in various

ways. I'm glad you brought that up because one of the other questions that came up in my mind when I was reading your book was, do we need a new kind of international body to think about these kinds of agreements around AI, or are organizations like the UN sufficient for this?

Well, as someone who's worked a lot with the UN, no, the UN is not sufficient. On the other hand, there are bodies like I work with the IETF, the Internet Engineering Task Force. They actually specify what all those routers and connections do. And they're open to doing this. They're a little slow, takes them a few years to do things, but they recognize that we just got approved for a group to actually redo the structure of the internet just a little so that we can keep track of what AIs are doing. We have the World Trade Organization, which has its ups and downs, but basically you need to have compatibility between different countries, otherwise you can't have trade.

But how feasible is that for AI?

Well, the reason the WTO exists is that every country on Earth needs trade to survive. Okay? And so they say, well, here, buy my widgets, okay? Or whatever. And the other country says, yeah, but you don't have any quality control. You don't have any liability. You don't— so if you don't have that, people won't buy from you. And we're already beginning to see some of that, right? So anybody that's a trading Read, almost everybody, has to have something that hits minimum standards for liability, for compensation, for some level of honesty and transparency. The same thing we have today. When you buy something from another country, there are international standards about how the payment happens, What happens if the thing you bought doesn't show up? Maybe they're not perfect, but it's in everybody's existence to have that happen. And that's what organizations like the World Trade Organization are for. There's another one we work with, which is the fraud and terrorism task forces around the world. It's in their interest to cooperate because otherwise the bad guys move someplace else. And you have the same problem again.

Yeah, no, I mean, I can see that for like physical products and goods, but what does that mean for AI? What would it mean to have globally accepted standards or norms for AI? What would that actually look like?

I think that it's the best analogy is in finance and trade. So finance and trade is about everything, right? But at the end of the day, people support, send money on things, and sometimes get in disputes about it. Okay, well, AI is going to be the same type of a thing. It's a, it's a service that can be bad, can, can hurt you, uh, can be fraudulent. And, uh, so it begins to look a lot like buying, say, pharmaceuticals or buying, uh, uh, software production facilities or buying accounting from other places. You need to have an audit trail and you need to have liability.

Where does open source versus non-open source come into the picture? I mean, would this require that AI be open source, you know, as some of my colleagues are arguing?

A lot of it is. I mean, I think the place that comes in, and I see this because I work with many different countries, nobody Nobody who lives in another country wants to buy services from OpenAI if they don't have to. They want to have something they control. They want to have a data center that they control that are national. They want to have things in the local language that reflect the local culture. And with the sorts of open source things that are out there, which incidentally are almost as good as anything on the planet, you can do that. You can have your own and you can tune it to be your sort of local AI. We just had a major paper about that showing how you can most cheaply provide these sort of sovereign AIs, they're called, for every language in the world. And so people are rushing to do that so that they don't have to depend, using these open source software, so that they don't have to depend on just American or just Chinese commercial products.

Sandy, it's been a fun conversation. Thanks for your time. I really enjoyed this conversation. Any last words of wisdom before we wrap up for our audience?

Well, the reason I wrote this was that people are either doomers or lotus eaters when it comes to AI. They think it's going to be one or the other, and I think that's just not true. We need to ask what sort of AI we want for us. And once we have that sort of North Star, then we can build it and get there. If you don't have that sort of North Star, you're going to end up in places you don't want to be. That's the point of the book.

Thanks so much, Sandy. Well said. Great to have you on the podcast and look forward to perhaps catching up in person when I'm out on the West Coast next month.

That would be great. You take care, and thanks for having me. Thank you.