493. What Human-Centered AI Looks Like with Alex ‘Sandy’ Pentland

Professors FM.

You're listening to the Unsiloed podcast with Greg LeBlanc, produced by University FM. Unsiloed is a series of interdisciplinary conversations that inspire new ways of thinking about our world. So wherever you are, enjoy today's episode, and here's your host, Greg LeBlanc.

Welcome to Unsilenced. This is Greg LeBlanc, and I'm here today with Sandy Pentland, who is now a fellow at the Human-Centered AI Institute here at Stanford University. But also, you're the founder of the Media Lab at MIT.

No, I'm not the founder.

Not the founder.

I'm one of the early guys who was head of the department and things like that. I helped create it for what it is.

Helped make it famous, right?

That's right.

But you also had a whole bunch of other appointments at Yeah, and the Media Lab is an interesting place because it combines the study of media and information and design and human interfaces.

It's one of these things where it's asking about technology and society together. So most of engineering is we're going to build this sort of widget and design it. Most of sort of liberal arts tend to be we're going to look at the humans, but actually the synergy of them is the thing that's driving a lot of our good things and bad things in society. So you have to ask, how do these things fit together? What would you design for people and how do people have to change? And those sorts of questions.

So it's by its very nature interdisciplinary. And I think you founded this project, was it called the Connection Science Project? Yes, that's right. Which also was incredibly interdisciplinary.

That's right. And not just at MIT, it's everywhere in the world.

Well, I guess it's—

Sparsely everywhere in the world.

If it's about connection, I guess it would be difficult to— Sparsely, but yeah. Yeah, you can't really draw lines around it. But also you've written a bunch of books. The first book is called Honest Signals, and then this one, Social Physics, which is now, gosh, 10 years old at this point.

Yeah, it's amazing. Yeah. So the new one is called Shared Wisdom.

Ooh, okay. And when's that coming out?

Well, I will give you a copy in probably a month.

Okay. All right. Maybe we'll have to come back and have another conversation. But you've also written a bunch of other books or co-written or co-edited a couple other books. I have some here with me. Building the New Economy, Data as Capital, and also one on fintech. And part of the story in the fintech book is this notion of Distributed ledgers and decentralized finance and—

The new economy one too, yeah. And that also connects with things like having a model where there's these big hyperscalers that own all the data is not seen as a good thing. And there's a lot of pushback about that. And people are coming up with things that are distributed where folks have much more control over their data, which gives them much more sort of advantage for things. And that's part of the trend. A thing that drives that is security. If you put everything in one spot, it's called a honeypot, and all the bad guys go for it. If it's distributed, you'll win the occasional one, but you're not going to win everything, so it's much safer.

Now, you coined this term. Did you coin this term social physics? And the analogy that I like about this idea of social physics, you said that traditional physics is about the flow of energy and how it enables action, and social physics is about the flow of ideas. Although I think you define ideas broadly enough to capture things like beliefs and preferences and even routines. Routines is something that we in the business school are focused on as the kind of unit of analysis.

Yeah, social physics was actually, as a phrase, invented in the very early 1800s in France.

The Comte?

Yeah, for instance. Okay. Right? And it's where we got the idea of a census. We're gonna actually collect data about things and ask what's happening. It wasn't about interpersonal connections and idea generation. It was about demographics. But you have to remember that back then they didn't know how many people were in each city. They didn't know how many people were young, old, how many people were dying. It's only when you saw lots of dead bodies in the street that you realized something was wrong. And so the idea of sort of quantifying human relationships and human living got started way back then. And today, of course, you think about how do I design my company and my society, my media to be able to leverage the way people are to get better results, to make it more

And the big thing that enabled what you were doing at the time was the availability of all this new data. Yeah. So the difference between traditional social science where you're doing these observations maybe in the lab or maybe you're stringing together some anecdotes or even the census is something which is, well, it's very labor-intensive to collect all the data and it's not always completely accurate. Yeah. Swallowed up in a world of sensors. And so we can do stuff that we never even imagined before, right?

Yeah. I mean, the proliferation of cell phones and online buying and things like that means that you can begin asking about patterns that you could never ask before, which is both good and bad. And the key thing is this idea that we started called computational social science. So we had this big event and the science paper about it. And it's really the idea that You may measure these things, but it's the relationships between them that tell you about causal connections, or at least potential causal connections. So you say, oh yeah, he has a lot of friends, but are they distant friends or close friends? Do they talk at work or not at work? Those are the sort of questions that give you insight into what's going on in the social organism that we call human or human communities or human groups. And that's what We started when we realized there was all this data. We started it incidentally because at one point it was clear there were going to be things like cell phones. So we built some before there were real cell phones and we put 'em on people.

The badges, you had the badges.

The badges. Yeah. That's right. That's the Honest Signals thing. And what we found was there were all these patterns that people sort of knew, but you could actually watch how groups behaved, where ideas came from, who were friends with who. There's all sorts of things that come out of the pattern of interaction, and even more so in the pattern of how people speak to each other. Are they convincing? Are they not convincing? And there's amusing stories about this too.

Well, I want to dig into that research, which I do find fascinating, but some would argue that this term social physics is a bit of an oxymoron. I mean, I teach a course on behavioral finance and I oftentimes teach it to financial engineers, and the financial engineers have a very physics-like view of behavior in markets. They're the ones that, I mean, apply Brownian motion to the pricing of derivatives and so forth. And the behavioral finance course in a way is trying to push back against that. Now, part of it's using terms, using feedback loops and so forth, which is consistent, but then there's this element of animal spirits that seems to go against this notion of a physics-like model for human behavior.

No, I mean, first of all, turbulence, things like that, weather, I was just watching something about tornadoes. Those are not equilibria phenomena. Those are these big things that happen, and those actually turn out to be the most important things in the evolution of the economy, like overinvestment in railways. Well, almost all of them went bankrupt, but now we got railways. We're probably seeing overinvestment in AI, but we're going to have technology to do that now, even though most of those companies will probably go bust. So The social physics, you mentioned people using Brownian motion, and I actually am an advisor to a large sovereign wealth fund, so I sort of know something about Brownian motion is wrong. That's not the right model. And the fact that it's not the right model has consequences. Like for instance, the 2008 crash, that was because they used models like Brownian motion rather than the correct models, which are very long-tailed and spiky and things like that. And so when By coincidence, mortgages in several cities went— the whole math blew up.

Yeah. The correlations all went to 1.

Yeah. It was bad, right? We all know that. But that's because they didn't have a really good notion of what is the physics of how ideas spread, what causes people to be convinced and take action. Those sorts of things matter because at the end, finance is all about people making decisions. And so if you know what's influencing those people, Then you can make a much better choice of what's happening.

Right. And so at the core of your work is the diffusion of ideas. And I think some people might have a naive notion that making the diffusion of ideas is always going to be a good thing, making it easy for ideas to disseminate in the hopes that the good ideas will always flow more rapidly than the bad ideas. The bad ideas will go extinct. I think a lot of people have a Darwinian idea where you have variety, selection, and inheritance, and it's the fittest ideas that kind of bubble up to the top. I mean, I think you're telling a much more complicated story than that.

Yeah. I mean, there's a bunch of— it's not that the Darwinian thing is wrong, but there's a lot of things around that, right? Like for instance, people are not real smart. Even you and me and all the people listening, we can only entertain several things at a time. And actually, if you look at financial decisions, you've got a number of like 3 or 4 strategies at any one time, which means you can't consider all of the options. And what people do is they get stuck. And if the people they interact with have the same 3, then that's just what everybody does. It's the received wisdom. Your retirement account should be 60/40, 4% a year, all those. Those are the sort of things everybody just knows those. Now, are they true still? Suspect not. And that's because you get these memes, sort of new ideas that pop up, When you get a local concentration of them, people accept them as received wisdom and begin to act on them. And it's not sampling the whole space of things. People are also not able to entertain complicated things very well. So what you get is you get these concentrations that are bubbles or panics, and that writes a lot of history.

So the idea of the wisdom of the crowd only works under certain conditions.

That's right. It only works under certain conditions. You have to have a good sampling of data, all the different circumstances, If you're gonna learn with other people, they have to have the same utility function as you. I mean, if they want things to go this way and you this way, then learning from them is not gonna help you, right? So you have to be able to really judge whether what you're learning from them is useful to you also. Okay? And there's some very simple statistical rules for doing that. In animals, this is called foraging. Animals go out, they try different things. Some of them eat the poison berries and die. All the other animals look at that and go, oh, Okay, not going to do that. And the same is true in our lives. Some people go out and invest in X and X blows up and everybody looks at it and says, yeah, those look like bad investments. And if you get enough of that, then people have that as a received wisdom.

So in other words, imitation is good up to a point, but if there's too much imitation or the imitation—

If the imitation is from people who don't share your goals,

Well, you can have the same goals, but you see that someone is using an approach perhaps that appears to work. If everybody converges around that, then all—

That's probably really bad.

Then all the heterogeneity goes away. That's right. Then you get some kind of herding.

You get herding. I mean, a couple of really basic things is that we live in a non-stationary world. So most of the things people are taught are about equilibrium, like most of economics is around equilibrium. We are not at equilibrium. I'm sorry. And the important things are the non-equilibrium events, not the equilibrium events. Similarly, a lot of things are long-tailed. Anything that involves copying people or biological things, those have these rare events that happen. And those rare events tend to dominate the total returns or the total outcome. And if you're using normal logic to counting votes, something like that, you miss those and it's bad.

Well, look, my field is mostly about organizational design. Mm-hmm. How do you build an organization that's likely to thrive and innovate and so forth? And what I often observe is that within organizations, you have these little pockets of insight, pockets of knowledge, and there seem to be frictions when it comes to getting them from the leading edge to the kind of— Yeah. Rear guard. And it's very much like you have a country which has a famine in one region and then a surplus in another region, and there's something wrong with the trains. So what is it that Impedes the transmission of good ideas from one part of the organization to the other.

Organizations are always today siloed, and they're siloed for a variety of reasons. One is that we're not so smart. We can only understand a certain amount really well. And when it gets more complex than that, we go create another silo. We break it apart and create another silo. And people don't like to share things out of their silo because that means they're not valuable anymore, right? You like to control the stuff and believe that your expertise is valuable. And if someone comes and says, well, gimme all your data, That's like, hey, we're going to fire you in a month, right? People don't like to do that. And the data that they have is not contextualized. It means something to them, but somebody else looking at it will say, oh, I don't know, it's a bunch of ones and zeros. What does that mean? You need to know what it's measuring, why. The intention is usually not in the data, it's in the context around it. So this is one of the places where AI, today's AI, can be really important. 'Cause what AI is, is looking at all the stories that people have told and saying, well, here's this sort of big pile of stories. And if you wanna know about some things, here's what people say about it. Okay? Now, unfortunately, the way it's designed today, it's all the stories. So all the stuff on 4chan, all this stuff that comes from all the crazy people, that's in there too. But if in a context of a company, it's pretty interesting. Because what you're talking about is what's common practice at this company. If you say to one of them, well, I have this sort of position and this sort of thing, what do people typically do? It could actually give you a very good answer. Now, you should check because sometimes it says weird stuff, but it gives you this context for what you're seeing. It's very hard to learn unless you've been there a long time and things are pretty stable. And this is why these sort of AI buddies are so good at helping to onboard people. They're giving that sort of newbie who doesn't really know where everything is and what people typically do, it's giving them some sort of insight about at least what other people say they do, maybe not what they actually do. And that means that you can be a lot more sophisticated about how you network people together and begin to sort of melt the silos a little.

Yeah. I mean, what I find interesting is if you do have some good ideas somewhere in the organization, one way to approach it would be to have sort of a centralized switchboard where the best practices, the good ideas and so forth get routed up to somebody at the top who then redistributes it and pushes it back down to the parts of the organization that lack it.

That's the way most organizations are designed.

But the other way is to sort of create these horizontal connections where you get these folks to sort of provide some lateral transmission of this knowledge. And the way you would do that is by orchestrating connections. When should you think about using the one approach versus the other?

Well, I don't think they're directly in opposition. I mean, routing it up and down has clear problems because there's limited bandwidth up and down. And so you're very unlikely to get things that are specific to what you're doing. The peer-to-peer thing, Well, that might work if you had the right people. So some of the studies where you have the best sort of productivity in very high technical areas really emphasize that. You find the experts about this and you have coffee with them once a month and so forth, but you have to find the experts and you have to maintain that relationship and identify things that you think are important that they may not think are important. One of the interesting things about the new sort of large language models is that It can give you sort of a sense of what people are saying around a subject. You have to query it, you have to show it what you're doing, you have to have a discussion with it, a Socratic dialogue. What do you think people do? Why? Who does that? But within a community, you can do that because everyone's willing to share at least the public stuff that they do. I mean, the stuff that's not private is the business stuff. So the studies that have been done, which are Not so thick on the ground yet, but the studies that have been done show that that actually works really well, as opposed to asking the AI to solve something or to do something. That typically doesn't work very well at all because occasionally it makes really bad mistakes. And even more important, it's sort of like an average of what everybody says, unlikely to be good.

Well, if you're trying to come up with new ideas or innovate some new product or design an organization that was more likely to adopt and generate good ideas. Would you want to create some silos? I mean, it seems like there are benefits to having teams that spend a lot of time talking to each other, but then there's also benefits to connecting these more specialized teams in some way, right?

You need teams to be able to come up with enough of a concrete plan that you can get collective action. That's what communities, friends, et cetera, they're about trying to sort through all the noise to find something that you can all agree on and do. The longer connections are so that you don't end up in a rut doing stupid things. You have more context to what you're doing. So this is where you need to design not just the people and the org chart. I mean, first of all, it'd be good to get rid of the org chart. Well,

there's the formal org chart and then there's the informal org chart.

Right. And so the question is, how do you engineer the informal one?

Right.

Because that's the one that really matters. And I'll give you 2 stories about that, okay, that I think are really interesting. One is a recent conversation I had with the CEO of an organization that has about 350,000 people, operates all over the world. And what he said is, is he's going to be using these AI buddies to help make the organization more agile. And the way he's going to do that is have a core of people It's much smaller than the 350,000, more like 150,000, something like that, that are in the office. Another 100,000 or so that are hybrid, but then 200,000 that are gig workers in a sense that used to work at the company. Mm-hmm. So they know the culture, they know sort of the value system. And what these sort of AI buddies are going to do is help them know what's going on and what people are currently doing in the corporation. And I think that will work. I think that will work pretty well. Unlike earlier attempts at this, like all the big companies you can imagine, Microsoft, IBM have tried to do this, but they relied on people typing things in to capture the knowledge. Now you can capture that knowledge passively just from all the things that people do, and you can report it in a sort of a sensible way. And that makes all that difference for them. The other one that's interesting is the US Army. Yeah. So the US Army went into Iraq, you might remember, and got its butt kicked because they were facing people who were well-armed, well-organized, but local experts.

Right.

And they had the top-down, up and there, it just didn't work. They were always 3 days behind. Okay. So what they did, this is General McChrystal did, is he reorganized things so that The power of operation was now down at the leaves. So local leaders now could actually do things. But to do things, they had to bring in all the stakeholders. So the different types of things, different branches that would normally not talk to each other. So transportation, logistics, this, that, and the other thing would all come together to decide what to do here.

Within a small geography.

Within a small geography. He calls it team of teams.

Mm-hmm.

So these people are members of possibly 100 teams, but on a task-by-task basis. And it's the local operations that decides what's the task and the problem. And then the other thing that he did, which was really interesting, and not just him, this is sort of Pryor's, decide that orders were not orders. There was this, you'd get an order, say like, oh, take this thing. But there'd be also this document called Commander's Intent. And what you really had to do is what was intended, not what you were told to do. I mean,

that's the kind of organization most people want to work for.

Exactly.

Because all of us are stuck trying to fulfill some ridiculous order.

That's right.

When we know it is inconsistent with the purpose of the order.

That's right. Exactly. And so what he did is gave people cover to take the best action that they saw, as long as it was roughly aligned. And that's another way of inducing this sort of agility, which you need for spatial things, but you also need it for new technology. So you talk about silos. The real thing about why you might want to create silos is to try things out. So people always talk about having testbeds or things like that, and it's enormously difficult if it's a top-down organization. I've got it.

Particularly if you're trying to develop some kind of norm or shared cultural belief, it's just going to get diluted to homeopathic potency if it's within this much larger organization.

So if you have this more sort of distributed thing where you sort of understand what's supposed to happen and you also have the official empowered to actually try stuff out, then you can be a lot more experimental and you can share that. A lot better than if it were all top-down. So that's 2 stories that I sort of like because they're taking these very staid organizations and they're breaking them up to be much more agile and much more experimental. And that experimental is really critical because one of the things about how people think is we are not logical. So you're talking about how we make decisions. We're actually pretty bad at it. That's why we go to college. I have a book that says 256 common errors people make in syllogisms. So that's saying 3 lines of text, short lines, and there's 256 ways

to

screw up regularly, right? Where

do I get this book? I gotta get one of these.

Yeah, exactly, right? So the way you have to do things, and this is actually the way science works, is you have these hypotheses, that logic, right? So it has to be this way. But you try it out and you find out you're wrong, but something else works. And it's that trying things out, hypothesizing, trying things out that gets you to stuff that actually works. So you have to measure what happened. You have to have a little bit of logic and context knowledge to sort of come up with hypotheses, but you have to have the freedom to experiment too. Otherwise, it's all going to be just this big machine operating from first principles to be able to, and that rarely works.

Well, I want to go back the research you did on organizational information flows. And I use this research in my class. I used to teach a class on the workforce and workplace up at Berkeley. And most leaders know what the org chart looks like, but they seem to be— they have a hunch that that's not the way things really work. But they haven't until recently had the ability to get some objective view of how things work. We can kind of do that now. That's right. I mean, because of the digital footprint that we have. But if you focus only on the digital, then you miss this important face-to-face stuff. So I think that's what you were trying to capture by combining email data and text data with this badge data. What's the state of the art there now? I mean, if you're a CEO, can you summon your CHRO into your office and say, give me the lay of the land, let me know how things are working here?

So the state of the art is that there's a battle between Privacy, personal ownership of data, and corporate ownership. So that's one thing. People abuse this in bad ways. You don't need badges anymore because, for instance, you badged in to get into this. Well, we know you're here.

Right.

Right? And both of us have cell phones that sort of talk to the router here, blah, blah, blah. I mean, it's not quite like the badges, which would give you body language also, and it would give you prosody. It was like, are you upset or are you just bored or whatever? You get that off the biologists, but you get a lot of it. What are the patterns of who's like meeting up and things? And there's ways of doing this that preserve privacy, but yet give you a sense of how interconnected your organization is. Are you siloed? Are you not? Are there people that are off and don't talk to anybody else? And you can do things like you can predict how ideas are spreading between different groups.

And presumably you could adjust the levers. So you could say, hey, this group is not talking to this group as much as I think they should.

That's right.

And then you do things to either increase or decrease the—

So you're analyzing what is probably the most important part of the organization, which is the information flow between the ideas and the communication and coordination. And if you see parts that are not coordinated, you might want to ask, should they be coordinated? And if they should, then you want to change things. Like one thing, we had a bank, we actually had all the desks moved around so that people ended up talking to each other. At the water cooler and things like that. Fixed a lot of the problems because, yeah, they got the emails, but who reads emails that carefully? And all this sort of implicit knowledge was missing because they didn't have the sort of informal interaction.

Yeah, my favorite study of all of them is the coffee break study. Coordinate the coffee breaks and bam, right? You get a huge bump in productivity. I mean, who would've thought it?

Well, if you think about the organization as a sort of information machine, of course you'd think about it because there's all this stuff that you have to trade and you have to know the context of things, which means you need this sort of informal conversation that's not about a particular project. It's about how are things going generally, things you might want to know about the whole situation, just gossip basically, but corporate gossip typically.

Well, it also builds trust. You talk a lot in the book on honest signals about how much of the transmission of knowledge and beliefs and norms takes place In these sort of sometimes nonverbal ways or nondeclarative ways.

Well, so yeah, I was running an organization in India actually that had all these very famous people on the board of directors and they were terrible. And the reason they were terrible is they all would say things with such passion that everybody just went, oh, that must be true.

But that means it's a dishonest signal, right? If you can fool people.

Yeah,

yeah.

Well, I'm They were using it because people would read it as if it were true. Okay. So the normal way it works in daily life where you're not trying to deceive people is honest, but people who are very sophisticated and talented can manipulate it. Yes.

Right. Because I mean, you were able to predict the outcome of negotiations. You were able to predict the outcome of pitches, right? For funding and so forth. And so the whole time, of course, when I read the book, I was thinking, oh, Well, we just have to teach all this stuff and then everybody will be able to give a compelling pitch, but then the value of the signal would be dissipated.

People say that, but I think that's not true. I think what happens is it becomes more stereotyped. It's like manners. Hi, good to see you, shaking hands. Why do we shake hands? Well, because 1,000 years ago it meant you didn't have a knife in your hand or something. We don't really know. So what happens is people get to be signaling in ways that other people understand. Now, they will use it for deception sometimes too, just like you use manners for deception. Oh, you look lovely today. Right? Yeah. God, why am I dressed that way? That sort of thing. But I think what ends up happening is people develop patterns of communication as they become aware of the signals, and we see that.

Well, and you point out that if I'm exposed to a set of ideas and beliefs, I'm more likely to start to agree with them and adopt them and so forth.

Yeah.

And that's also going to be stronger if I'm exposed in person.

Yes, much stronger.

Yeah. So now with so much less of this interaction happening in person, is this reducing trust levels? Does the impact of this transmission, is it kind of watered down? I mean—

I mean, there's a whole bunch of things happening at once, so it's hard to say exactly what are key causes, but it's really clear that trust levels are way down. on all sides. And that when you have face-to-face communication with people, it can be much better. Now, not always. You can also end up hating some people, right? It just means that you have a much more established relationship. Okay? So getting together to talk with the other side doesn't always work. Sometimes it backfires.

When you meet these politicians or these CEOs, sometimes you meet them in person and you're just like, I mean, sometimes I'm just so impressed by their magnetism or whatever it is. And you don't really see it on the television screen as much. And you certainly don't see it when you're reading in the New York Times or whatever. And so presumably a lot of what they're able to accomplish, they accomplish because they have—

They're signaling all these sorts of attitudes and who they are.

But in a way, I mean, if you're on TV, it's hard to think that they're looking you in the eye, for instance. But when you meet them in person, there's all this—

So just in general, you can predict trust levels by the patterns of communication independent of the signaling. Now, I don't mean it's— the signaling conditions the patterns. If you sound like an idiot, I'm just not going to call you up. Right? Okay. So there is a signaling component in it, but that's underlying it. When we looked at large numbers of people in an organization, And then later ask them about trust questions. So you don't ask people, do you trust that guy? 'Cause there's a—

Yeah.

Yeah, right? But you do ask operational things like, would you loan him this? Would you give him a break for that, et cetera, right? And what you find is when you have mutually reciprocal communication, in other words, it's worth my while to call you up or talk to you, and it's worth your while to seek me out and talk to me. Well, what that means is we mean something to each other, right? And it's good enough that they're willing to put effort into it. And that actually is trust. It's this sort of expectation that there's something valuable for both of us actually in the future because of the history that we have.

Now, when it comes to these institutions, a lot of people say, oh yeah, I trust Amazon or I trust Google, or I trust OpenAI.

I think that's a real misuse of the name— word trust.

But do people have sort of a personification in their mind, do you think, when they say that? They're thinking Elon Musk or Sam Altman or Jeff Bezos or—

I— it's hard for me to know exactly where that misuse came from, but what they really mean when they say something like that is, this will reliably be the same way it was before. That doesn't mean that these, I trust them. They may be slimy bastards, but every time I talk to them, they're slimy bastards. So I trust what's going on to be slimy bastards. Trust the way it means between people is I trust them that if things don't really work out, they'll still cut me a break. They got my back. They're really trying to help. That's the sort of human type trust. And then it's been perverted in various ways. by politicians, by technology guys, et cetera, to mean it operates according to rules and it always does the same thing.

Right. But they are trying to create these personas that will be your interlocutors. A lot of the AI companies are thinking, well, if we can create a little, I mean, not like Clippy. I mean, nobody trusted Clippy, but if you had sort of an assistant, a virtual assistant who would be the person you would talk to, they would have a voice and so forth. Are they thinking about the human-like characteristics they can leverage to get people more comfortable with that?

Well, yeah, I know. Clearly, so this is this whole sort of agents, AI agents, agentic systems, things like that. And you're going to see this confusion between reliable and trust really come to the fore. I mean, what you want is something that is going to advocate for your best interest, where it understands your best interest and what you want. It has to do that. It can't just be A thing that doesn't know you in some sense, in the relevant sense for this transaction. So you may value some things more than other things. If it's actually representing you in something, it should know that because that's the trade-off you would make. Okay? So it really has to be personal, which means that the data that it's running on can't be owned by Google. It can't be owned by Microsoft. It can't be owned by Amazon. It has to be something that you own or a fiduciary like your lawyer. Mm-hmm. your doctor actually manages for you because it's very personal. I mean, what are the real trade-offs that you want? It has to represent that to be genuinely trustworthy.

Well, I mean, when you go to a doctor, obviously you're hoping that they give you all sorts of good advice, but you also are expecting to get some sense that they really care about you. And a big part of the therapeutic experience is this stress relief you get from Knowing that you're in good hands. But if I go to an AI and I ask for a diagnosis, and then I ask for a treatment plan and so forth, I presume we could do this without ever encountering someone with a white lab coat, except when perhaps they need to inject me with something.

Well, the typical way our society works that way. So in Europe, they would say, no, you just don't get to do that. And if somebody tries to do it, we throw them in jail. Okay? In this country, they say, well, you have to have disclaimers, and if you screw up, we're going to hit you with a billion-dollar fine. And so it's the liability that drives a lot of the cleanliness of this sort of behavior. That's post hoc. I mean, they already had to screw up. But it is something that, you know, it works with car brakes, it works with— everything has this UL on it. That's because people used to get shocked by the electricity. Right. And this not-for-profit Underwriters Laboratory Well, no, we're just going to put a— we're going to test everything and let you make a decision what you're going to go with. And so basically what they're trying to do is trying to up the level of play in the society by putting in knowledge there and by having entities like underwriters or Consumer Reports, a group I work with, who are really trying to work in your best interest. They genuinely are. So you could see That's their business model. That's what they profit, and that's what they collapse by if they're not doing that.

But I mean, the beauty of this, if we have a centralized database with all of the medical knowledge known in the world that you can tap into, I mean, the beauty of that is that you're not going to wind up with some third-rate doctor giving you outdated advice. There's a lot of evidence that says that if the doctor got into her training 30 years ago, you Old treatments. If you go to a country that has no doctors, you're going to get God knows what kind of advice. And so with these tools, the minute some new insight emerges from the scientific literature, it gets diffused almost instantly.

Many things to say about that. Okay. One is, you know, in medicine they do meta studies. Do you know why they do meta studies? Because the studies are not reliable.

Sure. Yeah.

Yeah. Okay.

But still, once it becomes accepted best practice, we don't have to wait the 100 years for the scurvy cure to make it to the nether reaches of the world.

And then the second thing is that the business models of most medical things prevent sharing of data so that you have a hard— for anything except the most common diseases, they don't see many people with that disease, which means there's no sharing of data, which means that you're going to get substandard stuff. No matter where you go. Okay? I mean, you might accidentally luck into the one that's good, but they don't even know that they're good. Okay? So what you need is you need some way of sharing data between all those little medical silos that doesn't threaten currently the profit motive of those people or the privacy of the patients. And my group does that sort of stuff. It's really advanced cryptography, but you can now do that sort of thing. It's just the regulation and the business models are such that— They're not much interested in it because they're doing just fine, thank you. So there's a bunch of things there. What you were really asking about is the AI advisors. And so in a situation like that, all those things I just said, it means the advice it gives you is not reliable. It's not unreliable. It's just you had better check 3 times with 3 different people. And then in the end, you're going to make your own bet because— There are no reliable medical procedures or things. They change over time. New things come along. And so if you're going to give people advice, you have to tell them, this is the best I know, but it may not be correct.

But I think you made another point, which is that if those best practices diffuse too rapidly, then it sort of eliminates any of the heterogeneity that might allow you to advance the knowledge. So you wrote in praise of I don't know what you call 'em, stubborn people, or like, you know, people who just refuse to go along with the consensus.

Yeah. Yeah.

You know, you need some people or people who, uh,

willing to experiment with it. Yeah.

Right. Like who, who's ever gonna launch a lawsuit with a very low probability of success? Well, people who are just super angry and you kind of need those people.

Yeah. Yeah.

In order to advance the common knowledge. Right.

Yeah. So today what we have is a situation where the practice of medicine is heterogeneous across the world. And it's likely to stay that way for a long time, even with these AI agents, because the people in this country are actually genetically, food-wise, exposure-wise, a little different than the people over here. Maybe the drugs work differently, right? And there's all these sort of contextual variables, but you'd like to be able to compare the two.

You'd like to track it. You want to—

You want to track it and you want to say, well, maybe we should try some of that because they seem to be doing pretty good. We're not doing it so well, right? So you want that sort of diffusion, but you have to protect privacy. You have to protect the sort of sovereignty of the local community, nation, whatever. And then you have to take the attitude that all medical advice is uncertain. In the end, somebody is making a commitment to something that is, it's not like gravity. Okay? It's not that certain. And you're in the moment, you have to make a decision because the person's sick. And there will be best practice, local best practice in that situation. Most people will go along with it. Some people will be stubborn. You want them to be informed and stubborn. You don't want them to be fools. And so it's a difficult problem, but this is the same sort of problem we have with dealing with almost anything in society. I mean, just to pick a common one, homelessness. How do you deal with that? Well, people try different things. None of them work all that well. We sort of haven't gotten the recipe yet. There's a variety of things like that we don't know how to do. The obvious thing to do, I think, is you have to let people try different things and share data about what works and what doesn't work. And over time, you can sort of build up a much better idea. It's not proof, it's not certainty, but it's a much better evidence-based way of figuring out what to do that's contextualized to the particular community or situation.

Well, I mean, that seems to suggest that the larger the community of knowledge, the more likely you are to discover better ideas. I mean, you talk about cities. The reason why cities get more innovative as they expand is not because there's more people as much as it's because there's more connections, right?

There's more communities that can learn from each other. Yeah, actually, absolutely. And the same is true on the large scale too. Originally, the states were called engines of innovation because they were going to be different and some of them were going to find good things that others could learn from. So yeah, you need the heterogeneity and you need the sharing. This is really, of course, what we have as modern-day science. I think this is where the Enlightenment did not come from the scientific method. The Enlightenment came from a bunch of people trying things, Writing it down and sharing letters with each other.

The Republic of Letters, they called it.

Yeah, right. Yeah, the Republic of Letters, men of letters, because they're all men. And you know, you got the Royal Society that would like write it down. And they also invented the notion of citations at the same time. So you'd write a paper, you were cool, so you got in. And maybe a lot of people liked that. So you could look at all these papers and say, God, everybody likes this one. And maybe that's a good bet. Of course, maybe it's not quite true everywhere and it evolves over time, but the citation mechanism is a democratic process whereby people who care about this show their approval or at least their interest in it.

What I find fascinating is how Amazon is organized very differently from companies of 20 years ago with smaller teams and then more connections. And so it's almost like the human brain with more connections and more possible combinations of different ingredients than you would get from an organization with larger teams.

So a way to interpret that is that the smaller teams are necessary to move fast. If you have a big team, it's really hard to get everybody educated and up and on the same page. And so, but with software and the sort of things that they do, you can do it with small teams. And so you can have small teams, which means you can get consensus about what to do pretty fast. And of course, the connections to everybody else means you can look for opportunities and learn from other people much better than other people. So it's sort of a win-win thing. You get people that are able to put things out there quicker. They're able to learn from all the other people that have done it better. And yeah, it works pretty well.

But is that because of hardware versus software, or is that because of the required speed of change?

Once you have a platform, then spinning up a new service or a new feature is pretty easy. So you can move very quick. So that's the happy accident. And that happens to work really well with smaller teams, et cetera, et cetera. So what you're gonna see with the sort of programming AI things, the programming assistants, is now becomes faster to spin something up. I mean, yeah, it's got bugs and stuff like that, but, you know, it's faster to spin things up than it has been in the past, which means the team size can get smaller. Mm-hmm. And if they don't have much more interaction with each other, then you're gonna get a whole bunch of incompatible things, which is not so good. And you're gonna have to shut things down and it's gonna be chaotic. And so I think in a certain sense, what these sort of AI buddies are gonna do, the programming assistants and stuff, is make most industries a lot more like Amazon. Hmm. Where you can have smaller teams that are more highly connected And you can spin things up, which means you can be more innovative and you can react quicker. So that's good for us, but it's also a little disturbing because there's a much higher rate of innovation. And a good example of something that's not very innovative is government services. And it's not just the bureaucracies, it's us. We expect certain things from government and that doesn't change very quickly at all. That changes over generations.

I just spent the whole day at the DMV, so.

Well, yeah, okay. Okay. Okay. And it's better than it used to be too.

It is better. I was able to make an appointment, which made my wait a lot shorter than it would've been.

Yeah, yeah, yeah. So one of the things this will do is up the rate of innovation, but that's going to be a real problematic thing unless we can do that with our governments and other sort of bedrock institutions, right? Social services, things like that.

Now, I think you're spending a lot of time speculating about the future industrial organization of these AIs. And I think you alluded to the notion that instead of everybody tapping into these commercial foundation models, we are going to have much smaller models. These things are going to be run on our laptops, they're going to be run on our phones and so forth. Absolutely. Okay. So what does a world like that look like? Because I think that the folks at Anthropic, the folks at OpenAI, the folks at the hyperscalers, This isn't exactly what they have in mind.

No, they're— well, you also have to look at, like, if you're Anthropic or somebody, not to pick on them, you're trying to raise large amounts of money. So you want to emphasize the fact that you need this large amounts of money. But when I talk to people in some of the more established companies, they're already building things that run on phones because they see that's where the real market will be. It's informed by the large models, but it's not really the future. So you see these happening. I saw one just recently that was a Chinese version competitive with GPT-4. The entire training cost, according to the people that said it, about $3 million.

Nobody's gonna trust that one though.

Oh yeah, they will, because it's the Chinese model, right? So I was at a meeting of CTOs. So these are like Fortune 50 CTOs talking about AI in their organizations. And first of all, The entire conversation was around how do you get the right data to the right people? And they were not even, they were just trying to get past their silos. They weren't trying to collaborate with other people yet. Yeah. But one of the things that happened is that somebody mentioned, oh yeah, and of course we have AI buddies for everybody. And everybody's like, yeah, sure. That's a real easy thing to do. You take all your internal documents, you stick it in a LLM, and you give it to everybody. And it's not a big model, it's a smaller model, but now everybody, all these organizations have just done this.

In like 2 years. It's crazy.

Yeah, right, exactly. And so what that means is that the evidence about this is that that allows people to be onboarded more quickly. It allows lower-skilled people to get more skilled. It generally improves productivity in various ways. It also improves sort of coordination. among the different parts of the company because now you can actually propagate things that work in better ways. So that may be the real revolution that's happening. Now, how does that happen out in the regular world like you and me when we have medical devices or medical recommendations? That is going to have to sort of work its way out. I think the only way it's likely to happen is you build things that have available all of these sort of data sources, not the actual data of course, but sort of results from all these different towns, countries, et cetera, and is able to present it to you in a way that you understand. There's a little bit of magic there that we don't understand how to do yet, but it has to be something that is a fiduciary. So fiduciary is a legal term. It means it is required to act in your best interest. What's your best interest? Well, Okay, it's got to know something about that. With medicine, it's probably pretty straightforward.

Well, in medicine, there's not just your interest. I mean, there's the interest of the public that doctors have to consider.

Correct. Correct. Yeah.

For antibiotics and—

And some medicines are very damaging to people. Their side effects are damaging, et cetera. So it's a really difficult area to get things that can give you advice. That's why doctors get trained so much is because it's It's difficult. I mean, the sort of hard reality is not possible to do a perfect job. And what comes as perfect changes over time and situation and person. And all that contextual stuff matters.

Well, I'm much more inclined to trust that than a drunken, tired, and stressed-out human.

Well, all the people that I know, and obviously that's a sort of not a uniform sample in the population, What they do is they look up things online, they look up several different things, you get a lot of BS in there, and then you talk to a doctor, but you're already pretty informed. And I think that makes it much more of a productive conversation than it might have been. The doctors originally didn't like that, but I think they're adapting to it. I mean, if you come in and you say, well, I read this paper or this paper, and the guy says, oh, could you send me that? Yeah, sure. Well, I mean, come on, this is one of your favorites right now.

Look, you talk about these connectors. I think it was in Social Physics, these folks who are the people who have this curiosity. And you said that they are oftentimes the people who wind up as leaders. And there was a little bit of ambiguity about the causal arrow there and the people who tend to be successful and tend to be wealthy, it seems like they can afford curiosity. And so I always wonder, Why aren't people more curious? Why isn't everybody curious? I thought this was like a human attribute, but it's in fact unevenly distributed. So what is it about these connectors? Is that a frequency-dependent strategy? I mean, does curiosity only provide an advantage in a world where it is relatively rare? Or if everyone was curious, would that sort of be bad for overall?

Well, so you can look at animal models who do the same type of Foraging. You go out and you look for other things, and occasionally you discover a great berry bush or whatever, or you discover that the lions live over there, go away. Those are the sorts of things that curiosity teaches you, but it takes energy to do that. If you're doing that, you're not feeding yourself, you're not feeding your kids, you're exposing yourself to more danger. So there's a cost, and this is known as the exploration-exploitation dilemma. You've got so much energy, how are you going to spend it? If you're feeling confident and like you're in a pretty safe place, you can afford to explore more. If you're in a really desperate situation where your exploitation just is not worth it, there's no more berries, then you got to really explore because otherwise you die. But that sort of trade-off is characteristic of any social species and in fact of most animals. And you see almost exactly the same thing in humans, and we've studied it in financial decision-making and so forth. And the things that you're referring to are things like sometimes people get in a state where they're satisfied and they need to do something else. So they don't need to explore because they're not being pinched at the moment. And so they'll do things that are more fun. They're working on social exploration or something else. If things get bad again, they'll start exploring. for making more money, higher status, whatever. And of course, so this depends on, do you perceive yourself to be in a good place or a bad place? Are you in a desperate place or are you in such a good place and you're ambitious and you sort of explore where you can afford to explore more, right? So typically young people explore more because first of all, they haven't got a lot of stuff going for them anyhow, because they're young. They haven't like built up a whole resume and things like that. And so they're in a situation where they have to explore more, but they're also more resilient because if they don't succeed at that, it's not the end of the world. They're still young and resilient and they have family and stuff to fall back on. The causal thing, all of this confuses the causal thing. Okay? Mm-hmm. So you can look at it in aggregate. Like you say, well, here's a neighborhood, there's a neighborhood, there's a neighborhood. And there it's very clear that the exploration and the connections really makes a difference over time. Takes a long time. There was a study out of Harvard by Raj Chetty and his team showing that intergenerational mobility depends on that too. If kids are exposed to people that feel like they're in charge of their future and they've been a little bit successful, if kids are exposed to that, then they will come out better than otherwise. It's a really interesting thing because the major effect is on kids less than 5 years old. So the little kid that you don't think isn't learning anything, they're just maybe beginning school, they see people who are, look, have it going for them and people that aren't. And if they can see some people like that, then they learn an attitude that helps them in later life. And what's really interesting is— For young girls, they have to see women like that. Seeing men like that does about nothing. For young boys, it's seeing men and women like that does about nothing for them. It's just really—

That's why moving neighborhoods can have such a big impact.

That's right.

But the optimal amount of explore behavior is a function of the environment.

Of your situation in the environment. Yeah.

And so, I mean, could we make any general claims about the optimality of our current levels of curiosity? Could you say, hey, this organization is not rewarding curiosity enough, or—

Oh, sure.

These individuals, given their life circumstances, are not curious enough, or do you think that we ever err on the side of too much exploration, or is it typically we have too little?

Organizations sometimes, in my experience, and I've been like a bored observer Telefonica and head of the data board at Google and places, AT&T and tech board at Google and so forth. Exploration has to have a purpose, right? It has to be, okay, there's something I want to try and find out or an opportunity. I want things to work better. I want to be more successful. Not just, hey, let's go have some fun and talk to people. And so I've seen organizations just say, oh, you should just go do Whatever you want. And that's often not very good unless the employees, the people, are determined to make their environment more productive, inventive, and things like that. If they're just having fun, it's not going to do very much. Most organizations err on the side of too little exploration. You have your job, you're supposed to do it. Don't be screwing around talking to the guys in sales. Okay. And

I think that's what Charles O'Reilly would say.

Yeah. I mean, it's absolutely true. I mean, even in organizations that are sort of famously inventive, like Google, it turns out that starting something that is un-Google is very hard. You have to like get the very senior management to say, no, we're going to try something that has a different way of behaving. And the evidence is, is that it doesn't work so well, right? I mean, how much of Google X has actually turned into things that you care about?

Well, last question. When this book was written, it was 2014. I mean, it was still kind of the early days of social media. And I think it was hard to predict at the time whether social media would lead to more connectivity or more isolation. And I think many of us were surprised to see how it impacted isolation. Do you think that with the rise of all of these new AI tools, it'll have potentially an impact on curiosity that would kind of stimulate it, but also it could potentially dampen it and eliminate it?

I wouldn't phrase it that way, but the phenomenon of people not meeting other humans because they're all on social media is real and disturbing, and AI can make it worse. Absolutely. That's not quite the same thing as being curious. They're curious just about things that are not actually in their physical environment or likely to affect them in that sort of immediate type of a way. The other thing that's wrong with social media that people didn't anticipate are the emergence of influencers. So social media, the business model was set up to be a rich get richer thing.

Yeah, it's not the long tail that we all thought it was going to be.

No, it's not the long tail. It's the concentration of things that are the loud voices. And you have to remember, people are, they have habits, They learn the habits from the people around them. They have a capacity to look at and think about maybe 3 or 4 things at a time over a period of time. And so what that means is our attention is not dominated by the long tail at all. It's dominated by the influencers. And if you look also at things like misinformation, it comes from a very small percentage of very highly connected influencers. That's almost all And so, well, gee, this is just as bad as the, like, not talking to people and sitting in your bedroom doom scrolling. So those are the two things that are wrong. And I think that a lot of this traces back, first of all, to the business model where it's the companies make money from these sorts of behaviors. So maybe they shouldn't. The other thing is that people who are influencers really, sometimes they're not making money off of it, but they're getting influence. for political, for other sort of things. They should be regulated like TV or like radio or something like that, where there's a license to have more than 10,000 followers. And if you are reaching that many people, you're going to start paying fees and you have to have certain behavior norms that you meet and we're going to check.

But I think, I mean, I was certainly an optimist about, oh, we'll see more connectivity. We wound up with a little bit of less connectivity in some areas. I thought we would wind up With long tail, all these wonderful heterogeneous preferences would flourish. And then of course we've seen this convergence with—

Well, no, don't be so— I mean, first of all, what the evidence is, is that people are much more aware of the long tails than they used to be. I mean, there's good statistics about that. So minority voices have been uplifted. I was talking to a fellow who got somebody in China to turn off the search algorithm. So that you just searched for things that had keywords. And what that meant was all the minority voices disappeared. Okay, well, you don't really want that. Okay, that's one thing. The other thing is most of Facebook is now groups. So the basketball team, the this, the that, the other thing. It's hard to argue that's bad. That's not like you have a million friends. It's like these are people who actually, in fact, if you look at the Facebook groups, They map very accurately to physical activities. So if I say, oh, you go to this place, then you're probably a member of this group. That's true. Or if you're a member of the group, you probably go to this place. That's amazing. So what they're doing is they're slowly maybe moving back to being something that's more a complement to physical activity than not. Just slowly. It's not done. It's not like I'm saying—

Probably not true for my Facebook data science Friends group.

And then the other thing, if you talk to people who've been involved in color revolutions and things like that, it's been very important in breaking oppressive regimes. It's been very important in some other sort of places. And the difficulty has been, of course, is that you can break things, but that's relatively easy, it turns out, but it's really hard to build good things in its place. So that's the thing that we're doing now here. That's why I'm here. Like we just published a thing called the Digitalist Papers. Everybody should get a copy.

I was at the event. I saw you there.

Okay, good. Yeah. So it's some of the leading thinkers thinking about how could we reform social media, AI, democracy to be the way we'd really like it to be? What do we want to do? How do we have to regulate things, reformat things to do that? And I think there's a lot of things in there that are particularly relevant. So One of the more surprising ones, for instance, is some of the people, the lawyers who are actually in charge of things like the national election, stuff like that, are of the opinion that regulating AI is going to be worse than AI. So, hmm, well, that's interesting because they think that there's going to be these blankets against certain sorts of innovation that are going to be leveraged against people and better governance and blah, blah, blah. So it's interesting to read.

Well, Sandy, so much to talk about, so much to think about.

Thank you.

Thanks for joining me and welcome to the West Coast. Thank you.

Thank you for tuning in to the Unsiloed Podcast produced by University FM. If you enjoyed today's episode, please give us a 5-star rating and review in your favorite listening app. To listen to our other episodes, please visit our website at www.unsiloedpodcast.com.