Sam Hammond on AI, e/acc, and Institutional Disruption

Institutions, when they're effective, have to have similar sources of external energy inputted into them. That's what FDR was, right? We had a system that was ill-adapted to the changing technological reality of the Industrial Revolution. And across not just the US but other countries, they had statesmen who were able to see from one local minima to another and make the leap in a way that wasn't possible just through gradual gradient descent across institutions or something like that. And so I think the US institutions have sort of fallen into a similar kind of like local maxima or local minima. There's a question like I get a lot. It was like, how would you regulate AI? I think the right level would be to say, you know, after transformative AI, what does regulation even look like? Does our regulatory state become like totally anachronistic? And anytime we've had the technology shifts, it's, it's, it's really shifted the balance of power between nation states and society. And I think this one is no different. And if you look at the printing press, like famously it was launched wars in Europe and challenged the authority of the Catholic Church. And I think there's like a sense in which AI is going to bring that instability and disruption to many more areas of the economy.

Today, my guest is Samuel Hammond. Sam is a senior economist at the Foundation for American Innovation, formerly known as the Lincoln Network, a think tank focused on bridging Silicon Valley and DC. Our conversation traverses frameworks large and small. We talk about pluralism, effective accelerationism, AI, and why we humans live at the knife edge of complexity from a universal perspective. We also dig into more practical matters like government dysfunction, social policy, and the think tank ecosystem. I think you'll particularly enjoy how adept Sam is at drawing connections between disciplines and bridging seemingly disparate ideas. So let's get right to it.

Sam, welcome to Upstream. Excited to have you on the podcast.

Thanks, Erik.

So by way of introduction, in the startup world, when we're looking to get a sense of an idea, we'll use a lot of X for Ys. We'll say it's like if Discord and Quora had a baby, or we'll say it's the Uber for X. And so I'm curious in terms of, by way of introduction for people to better understand you as a thinker, how would you kind of define yourself in a way that people could understand? Like, for example, I asked Sam Oberge if he's more of a, you know, in the Zeihan camp that geography and energy and demographics are what matters, or if he's more in the Balaji camp or that technology and human IQ are what matters, or if it's more of the Olson camp, it's just quality of institutions that matters. When you think about your worldview, what is the best sense for the audience to get a sense of how you think about things?

Yeah, it's funny you mentioned Samo because I think we might both identify as civilizationalists, that there's this, you know, there's nation-states, there's organizations, there's all these different levels of ways of analyzing society, but they all kind of culminate into a sense of civilization. And when you look across history at when there is broad flourishing in society, say like the Florentine Renaissance, it isn't just that they had like the best masonry and the best Banking and so forth. It's like they had the best everything, right? And if you look across the world today, like the countries that rank the highest in economic freedom and the Human Development Index, like they also have the best functioning governments and the least corrupted markets. And so there's like this X factor behind institutions that is deserving of study and it doesn't really boil down to geography or demographics. Those are major structural forces, but there's also these X efficiencies that matter for society as a whole.

So it's more of like an institutionalist, but more so like the people that make up the institutions? Or how would you—

Yeah, I think it's like an evolving science to try to figure out like why some civilizations succeed and whereas others fail and how they fall into dysfunction. Like broadly, you know, the other thing a civilizational perspective does is sort of, I think it's like the correct level of analysis if you're a long-termist. Right. So I've written a little bit about, you know, why I'm not exactly an effective altruist, but, you know, one of the critiques of effective altruism is sort of has this like God's eye view, like they're in control of the social welfare function for society. And I think really like what they're doing is doing morality for civilization as a whole.

Right.

And in some ways, like people have pointed out that like monotheistic gods, you know, arose with centralized governments and the kind of modern civilization. And so there's like an analogy between like this God's eye view of the world and the right set of norms and principles and decisions that matter for civilization as a whole. The issue is that civilization isn't a thing. It's not like a, it's not a unitary actor. Right. And so that's where the EA people kind of fall down is they think that they're, they're sort of importing this like God's eye view into an individual person. Right. And individual morality is something quite different. Right.

So you're, you're not effective altruism because you're, more of a pluralist, um, or you're more of a— you're more humble epistemologically.

It's like a gestalt, right? You, you can more, uh, moralize about the forests, or you can moralize about the trees, right? And if you want to say, like, you know, a, um, a central banker should, should think in terms of consequentialism because interest rates going up and down, it's going to affect a whole bunch of society, and you can't really, like, take that on a— you You can't really apply virtue ethics to that. Like you could apply virtue ethics to selection of a central banker. You want someone who's prudent, who has good judgment, who like embodies a kind of excellence in central bank management. The error is in sort of incorporating that God's eye view as an individual ethic rather than trying to see what is the individual ethic that aggregates up to that God's eye view.

Yeah. Just to make your consequentialist point. heard about the bank more clear. It's like the idea that in my family I might be a communist, but, you know, the famous Taleb quote, but in my town I might be a Democrat, in my nation I might be a libertarian or something like different sort of methodology, you know, sort of approaches might work at different scales or different scopes. And similarly, you know, the bank, you know, it shouldn't be a virtue ethicist, but maybe in different capacities you should be. Is that the idea?

Or that there's like a direction of fit. to kind of different kinds of moral propositions. And if you, you can't live your life as a utilitarian, you know, that you'll end up stealing your coworker's lunch to give to the homeless person outside.

Are you as big of a believer in the great founder theory as Samo is, that it's like individual people who make things happen?

That seems to be an empirical fact almost. There are clearly deeper structural forces that You know, there's always this like counterfactual thing where like, if Hitler wasn't born, would there have been a different Hitler? I wrote years ago a piece called The Bourgeois Virtue of Contingency. And, um, it was kind of a rejection of like these more Whiggish teleological theories of history where like, there's like necessity that it was going to happen one way or the other. Like, Yeah. there, that maybe on a long enough timescale, but like one of the lessons from Silicon Valley is Actually, it takes people with agency to really execute, to build things, right? Like Elon Musk's famous quote about like, it's easy to have the idea to go to the moon. The hard part is executing. And that often takes like very special people, especially in the context of a hierarchical organization where they can sort of manifest their leadership.

I always found you as an interesting thinker because there are not that many people who can, let's say, like, You know, talk to Balaji about technology or either the journalists or talk to the GMU crew about GDP, you know, or progress folks, but then also talk to Ezra about cost disease socialism and just kind of like, you know, we like meet them at their levels and variety of different groups that maybe are contrary to each other in different ways. And you seem to be one of the very few people who can do that.

Thank you, Erik.

You're weaving right now from social policy back into tech policy, AI. Also by way of introduction, why don't you unpack that transition a bit and share more about what you're trying to do and why you did it?

So I was born in Canada, but when I first moved to DC, it was to work at the Mercatus Center at George Mason. And that's sort of the hub of sort of rationalist libertarian types like Robin Hanson, Tyler Cowen, Bryan Caplan, that whole group. And I initially had wanted to go there because I had this interest in public choice economics and how institutions work and so on, but with that futurist asterisk on it. And I did a lot of work in tech policy. I worked on supersonic aviation, some stuff around drones. But in 2016, when I launched the Niskanen Center Social Policy Department, the reason for that was All my writing, all my recreational writing was sort of on how institutions evolved in response to the Industrial Revolution and how things like the New Deal, the kind of Bismarckian social insurance state that arose during that era, was a response to the forces of creative destruction and a need to compensate people to create buy-in for markets and stuff like that. And the subtext to all that work, and I've been working at Niskanen for the last 7 years was sort of creating the infrastructure, the human-level infrastructure for accelerating creative destruction to ensure that when people lose their jobs or when industries rise and fall, that first of all, they're allowed to rise and fall, right? That they're not captured by guild systems that build in social protection at the level of the particular market structure. And then that labor can efficiently recycle to new to new lines of productivity. And like I said, in the background was sort of this sense that I had since I read The Singularity is Near back in 2004, I guess, taking really seriously that you can actually predict quite a bit by just looking at Moore's Law and staring at it and trying to think about what it implies. And so even in Cost of Sea Socialism or these other pieces, there's always a line in there somewhere saying, We got to be careful not to bake in certain assumptions about the state of technology because we had the China shock that disrupted manufacturing across the US. The next shock is unpredictable, but it could be something like artificial intelligence or these other kinds of systems, roboticization, and that we need to be prepared for that. And now that AI is having a bit of an inflection point, I felt like it was the time to— I did my best to try to lay some groundwork to try to modernize unemployment insurance and all these things are totally broken in the US system. But now that things are taking off, it's time to get back to AI and how it may impact our institutions.

How do you view, we mentioned the Effective Altruist, you know, there's an effective accelerationist community that has merged it. And we're having, you know, somewhat of a tech civil war around sort of the AI safety people, the A people, and who say, hey, you know, we need to get regulation in early. This thing is, you know, literally paused. It's technology development. This thing is, this thing, you know, can get out of hand. This thing is very different from the past. And then we have the EA crew who are saying, hey, this is just a front for either for regulatory capture or actually, you know, the way to have alignment is to have all these sorts of competing AIs. And thus you want to encourage people to catch up. And, you know, we'll have competition the same way that nuclear capabilities have also been sort of de-risked by having multiple people have them. As you're watching this conflict emerge, where do you stand on it? How do you make sense of it?

Yeah, well, I'm now at the Foundation for American Innovation and we just had our launch party. And one of our guests brought a gift, which was that flag with the GPU. It says, from my cold dead hands. So I honestly have a lot of epistemic and moral uncertainty about what the right answer is here. And I'm skeptical of people who, I think a lot of the EAAC people, the effective accelerationists, they almost have this religious view of thermodynamics that we need to act in the interests of the heat death of the universe. And there's a certain internal logic to it, but I don't quite know where they get that sense of certainty. But the way I've talked about this in the past is that there are different dimensions of acceleration. There is the sort of horizontal dimension of taking GPT-4 and lower-level models and trying to build AI lawyers, AI doctors, AI accountants, and just full steam ahead. Screw the intellectual property holders. We should be able to generate Taylor Swift songs with a click. That whole copyright regime is now obsolescent, not something that we need to start building a surveillance state. So We know who's generating illegal Drake music. And then there's this vertical dimension where I think there's reason to actually be way more cautious. And I think that also came through in the recent hearing where Sam Altman was pulled before Congress. And there was sort of like the superficial agreement between him and Gary Marcus, who was the AI skeptic or naysayer on the panel. And, uh, but what Marcus would want, he wants like an FDA for releasing new models, even, even like simple models like AutoGPT or these like open source applications. And that is like obviously completely unworkable and would probably, probably destroy the AI ecosystem outside of big corporate labs. So I'm very much against that. But you can read between the lines of what Sam was saying. There was like this superficial sense of agreement, but what he was really saying was like, at some point, Maybe in a few years, we're going to have systems that are vastly more powerful than anything we've seen before, not just in terms of autonomous general human-level agents, but also things for synthesizing novel pathogens. And I think even the most ardent libertarian would understand why if you had a full-stack 3D printer that you tell it to make a novel pathogen and it will just do it for you, that that shouldn't be open source.

Yeah.

that there may be real societal risks there.

By the way, before getting back to AI, I believe you're a reformed libertarian to some degree. If you could tell, say anything to a libertarian that would help them have either convert or have a bit more nuance, what would you, or show them a blind spot they have, what would you say?

Well, I mean, there's lots of blind spots. I'm still like broadly libertarian. You know, I think libertarianism— I think there's a genealogy to our ideas and to our morality and so on. And sort of like the effective accelerationists where they sort of somehow get this like eternal cosmic moral principle or moral thing that they're optimizing for.

Hey, we'll continue our interview in a moment after a word from our sponsors. Hey everybody, if you're a business owner or founder like me, You'll want to know more about our sponsor, NetSuite. NetSuite provides financial software for all your businesses. Whether you're looking for an ERP tool or accounting software, NetSuite gives you the visibility and control you need to make better decisions faster. And for the first time in NetSuite's 25 years as the number one cloud financial system, you can defer payments of a full NetSuite implementation for 6 months. That's no payment and no interest for 6 months, and you can take advantage of the special financing offer today. NetSuite is number one because they give your business everything you need in real time, all in one place to reduce manual processes, boost efficiency, build forecasts, and increase productivity across every department. More than 36,000 companies have already upgraded to NetSuite, gaining visibility and control over their financials, inventory, HR, e-commerce, and more. If you've been checking out NetSuite already, then you know this deal's unprecedented. No interest, no payments. So take advantage of this special financing offer with my promo code at netsuite.com/upstream. That's netsuite.com/upstream to get the visibility and control your business needs to weather any storm. netsuite.com/upstream.

Some libertarians, I get the sense, especially from the natural rights tradition, think that these ideas were written into the cosmic, the fabric of spacetime or something like that. And this applies to libertarians, but it applies to everyone else too. It's really valuable to step back and try to step outside the flow of history and see the contingency of your ideas and how they arise in particular technological moments in history and not try to extrapolate them out of distribution. Because a lot of libertarianism ends up being kind of very formulaic. It turns out the world is just very messy.

Going back to AI, you have in one of your blog posts, you have a quote I want to read. The main risk for AI isn't it waking up and incinerating humanity, but rather the societal destabilization and potential state collapse that'll occur when 5,000 years of economic history gets compressed into a couple of months. How do we address this risk? And maybe adjacent to that, what can we learn from Mormons about how to handle technological revolution?

Yeah, one of the reasons I wrote that is And it's a little hyperbolic, I suppose, but maybe it's only 500 years of history that gets compressed into a few months. But first of all, there's a kind of inherent skepticism that the Yudites, the Eliezer Yudkowsky sort of school of thought on this, get when they talk about the dangers where he kind of solves for the equilibrium. Maybe in some actually quite distant future, will have such massive models that they'll just spontaneously learn how to solve the protein folding problem and escape from the lab and kill us or something. That seems quite implausible to me. But even if that is a world that we're approaching, there are intermediate stages that we will pass through, and those stages may end up being disruptive to the institutions that would be around to respond to that future risk. And when I look through history at printing press or the Industrial Revolution, these technological regime changes precede institutional regime changes. And so there's a question I get a lot, it's like, how would you regulate AI? I think the right level would be to say, after transformative AI, what does regulation even look like? Does our regulatory state become totally anachronistic because it is this 20th century thing That was built for an era of like standard setting bodies and everyone used to use USB-C and all this kind of junk. And that's the thing that I think is going to come on pretty quickly because a lot of what government work is, is like a fleshy API, right? And just like software is eating the world in Marc Andreessen's famous terms, software will eat the state. And anytime we've had like these technology shifts, it's really shifted the balance of power between nation states and society. And I think this one is no different, right? And if you look at the printing press, famously it launched wars in Europe and challenged the authority of the Catholic Church. And you can see analogies with information technology and the way it's disrupted traditional media. And I think there's a sense in which AI is going to Bring that instability and disruption to many more areas of the economy, right? So like right now the WGA is on strike because they want to get paid more for their crappy streaming reality shows and stuff. When Adam Smith talked about the Industrial Revolution, he noted that like one of its benefits was it disrupted the old guild system, right? And the WGA literally has guild in the name. And so maybe AI should be writing those telenovelas that are a dime a dozen because the human writers are already doing a ton of interpolation and borrowing and copying and pasting. People are trying to fit AI into existing modes of production where they're saying, oh, how can AI make the WGA writers more productive? Rather than saying, how is the entire system going to get turned over? And because there's been a Unprecedented degree of sclerosis and calcification and crystallization in our institutions. I think it's just going to be a very difficult transition.

Is the right model that it'll make the best people better, but that the middle class will just get decimated in whatever creative or caring field?

I hope not. I do think it's going to come for The upper middle class, kind of laptop class. I'm not excluded from that. And I think that could be quite egalitarian, but in a way that's actually egalitarian. Well, a lot of upper middle class people put signs in their window that suggest a degree of egalitarianism, but don't actually want to make sacrifices. And so if AI leads to much greater compression in human capabilities where people without linguistic facility are able to use an external word cell and bootstrap themselves up, that will level the playing field. But I do think there will be, as always, these Pareto distribution effects where there's the 1000x software engineer who's able to use the tools in a way that other people don't. But I think for the most part, it's going to be equalizing.

I think Sam Altman had a tweet that some bullshit jobs will be immune to AI because They were immune to, like, they were invented to begin with.

Yeah. You can't automate bullshit jobs.

Yeah. So yeah. What happens when the upper middle class, the PMC, whatever you want, like what happens when they're disrupted at a material level? They try to, you know, regulate like heavily.

Yeah. I think there's a Butlerian, we're on the cusp of a Butlerian jihad to end all Butlerian jihads. Right. Like, you know, the US Copyright Office is having listening sessions right now about how to prepare for AI in a world where the chart-topping Drake song isn't even by Drake. And that's only going to get easier and it's going to be very difficult to regulate because these are often open source models that can run locally on your computer. And so like one world would be let's build the architecture, some kind of blockchain tracking system that will know exactly who produced the music and maybe we need to have stuff on your GPU so we can turn it off if you make a Taylor Swift song without permission. So there's one world where we try to lock in the existing paradigm and have to architecture this insane totalitarian system to do so versus just letting the water wash over us and realizing that— Maybe this is the twilight of the celebrity and there will be people who gain fame and status because they're the best at pasting together things and making AI-generated music and content and so forth. But it won't be the Beyoncés. Their time has come.

The famous Peter Thiel quote that AI is communist, crypto is libertarian. You think it's either the opposite or it's more nuanced in that As you just gave an example, blockchain can be quite authoritarian or quite centralizing. And as you've given other examples, AI can be quite decentralizing in the access and power that it gives the average person, you know, that doesn't need to rely on, you know, big organizations. But of course, in the wrong hands, it can also be, or in centralizing hands, it can be used for centralizing ends.

How would you Yeah, no, I have said before that I would turn Thiel's phrase around. The one amendment I would make to it is rather than AI being libertarian, I would say it's potentially communitarian. And my intuition for that is if you take something like public education, public education serves a lot of purposes. It's a daycare, it socializes you, But the core, at least stated function of education is to train kids and give them human capital. And we've known for a long time that mastery-based learning and tutoring can have a 2 sigma effect on learning outcomes. The main problem has been it's really hard to have a tutor for every single student. It just doesn't scale. But I think in the next couple of years, we're going to see systems that are like the Miss Frizzle that makes eye contact with the avatar and the student and can recognize their voice and is adaptive and notices when they're struggling on fractions and does more fractions, but also has some reinforcement learning mechanism to make sure that it's staying fun and the student's engaged. And what that means is anyone with a web browser will be able to pull their kids out of school, And not just do a bare minimum homeschooling, but in some ways strictly dominate what they're going to get through public education. At the same time, if their jobs are being displaced and new jobs are coming into creation that are more remote or whatever, there could be a rapid unbundling of the public education system, sort of like what we saw during COVID but on steroids. Because during COVID 2 million kids left the system to do homeschooling. And so there's one version of that where that could be atomizing. But what I think will, what will actually happen is that, you know, after the transition, people will rebundle around like intentional communities potentially, or jiu-jitsu schools because, you know, you still need a gym teacher or chess clubs or things like that. And so AI will enable people to, it'll fragment society, but it'll enable people to like interface more directly with like people within their community.

You did your thesis, I believe, on religions and you've been fascinated with Mormons. So what lessons does Mormonism teach for how institutions can adjust to technological revolution?

Well, I mean, the first lesson is the fact that Mormonism arises out of the Industrial Revolution, right? So it's in some ways the most modern religion if you don't count Scientology or New Age. And it's this weird fusion of Second Great Awakening Protestant thinking, temperance, and like, you know, pro-growth, pro-technology with like the priesthood structure of like, you know, some, something like blending the Catholic Church and like Freemasonry. Right. And the surprising thing about the Mormon Church is not just that it's like one of the fastest growing religions, but the fact that it was born in that industrial context, it's very materialistic. There's a sense in which like God isn't omnipresent, but it's like a being that like moves at the speed of light, but it's still constrained by special relativity. So it has this weirdly physicalist underpinning to it. And that's also why you see in Salt Lake, for example, the Mormon Transhumanism Association. Joseph Smith said that we're living in the latter days, and in some ways he had sort of this Ted Kaczynski kind of insight that the Industrial Revolution was kicking off this techno-capitalist cybernetic system that would culminate in something crazy. And now it seems like that's prophetic, right? Some people think we really are living in the latter days and maybe the baptism of the dead, which is a Mormon practice, will reemerge as the replica that we create based on all our content to memorialize people. And maybe the far out long term is we all just upload our brains and become as gods. And there's some, some like literal sense in which, uh, the theology played out. But what makes it interesting institutionally is it, again, it is the fastest growing religion. It's been one of the stablest and most sort of like vertically integrated religions in the United States, whereas the mainline church has sort of collapsed and, and Catholicism is going through its struggles. The priesthood structure, the kind of hierarchical but yet decentralized structure of the Mormon Church, the LDS Church has made it incredibly resilient, right? And, you know, 60 Minutes did this sort of hit piece on the church where they're pointing out that they have like $150 billion in assets and stuff like that. And the Mormons, because they believe they're in the latter days, are like the preeminent preppers, right? Like Mormon communities have a bishop's storehouse that is full of foodstuff. Like every Mormon is supposed— supposedly supposed to have like 90 days of supplies on hand. And so it's like the cockroach of religions. I think if there's a really major institutional disruption, something like the LDS Church will stay standing.

While we're talking about religion, if Balaji were here, he would say that on the right there's a religion and it's Bitcoin maximalism. And on the left there's a religion and it's wokeness or progressivism. Do either of those meet the criteria for of a religion in your mind? What are they missing?

They're missing organization, right? Like, a lot of the problem with the mainline church, and you can think of like wokeism as just an extension of, you know, American Protestant social gospel teachings where like the Unitarians and the Quakers became so, you know, like, so pro— like, they became so Protestant they became secular, right? But, uh, but they're still like, they're still retain the normative commitments. But the big difference is they're not organized like they used to be. Church attendance is down. People are sort of acting—

Well, are they just organized in different ways? They just call them universities instead of— and by so Protestant, they became secular. And the commitments you're referring to are kind of like a duty to downtrodden, a moral equality, this sort of greater aspiration towards equality, and then this universalizing.

Yes, there are organizations, there are NGOs in universities, but in the sense of them being formative institutions, despite what some conservative critics think, I don't think universities make students woke. I think students are already woke in high school, and if anything, they make the university more woke. And there's a sense in which this form of Protestantism is an attractor like a sociological attractor. It's like the set of lowest common denominator things that it's very easy for society to evolve towards in a way that's spontaneous and doesn't require any kind of institutional organization or mediation. In fact, it's like the whole premise is like against mediation.

Yeah. And do you think the Joseph Henrich weird philosophy we've talked about on this podcast, that it's all pervasive and once the kids get internet, It's basically like pre-uploaded it and it's just a matter of time before their country also just gets more progressive. Do you buy that? Like that's just where everything's going?

Yeah, I wouldn't say it's, I wouldn't like color it by saying more progressive. It sort of imports like how it's manifesting in the American context, I think. But like in the Culture Series, the Iain M. Banks books, There's a line about the platonic dog. Like if you travel through developing countries, and I think this actually comes from Thailand, there's like this, this ur-dog, you know, any developing country where like dogs are running, you know, feral dogs are running around, there's, it's like this mutt-looking, scrappy-looking kind of dog that you see everywhere, right? Yeah. And it's in some ways like this basin of attraction for dogs that are interbred with a bunch of different breeds and they all end up looking kind of similar. There's something similar going on here where, but only it's like it's being mediated by discourse, by language. And one of the things language does, it exerts a kind of universalizing tendency on culture where if you have to reproduce your culture through words, norms that were particular to your community and were like implicit in the practices of that community become articulated as you ought to do X, you ought to do Y. And just because they're being transferred in that linguistic medium, it makes it seem like everyone ought to do X, everyone ought to do Y. And so, you know, Habermas has this whole theory of discourse ethics, which is essentially about this, the kind of pragmatic biases that language imposes on cultural evolution. And I think We're seeing that play out in the internet age where like, it's like logos working through history and, you know, the Iranians are protesting over the treatment of women in a way that, you know, they even have like a 3-word phrase like Black Lives Matter. There's a certain like, again, it's not like they're, it's not a psyop. It's not that they're like copying us. It's like there's a deeper thing that drives people when they're organizing in a way that's totally like disintermediated. and doesn't have institutional structure to settle on very simple slogans and to understand oppressor-oppressed relationships and stuff like that.

Does AI just accelerate all this just like any other technology?

That's a great question. It's really hard to know. It's really hard to say because if you take that thesis right, that the printing press was a prerequisite to the Protestant Reformation, right? Does this take us to a new kind of reformation? And my suspicion is that it does, but it may look qualitatively different. And in fact, one of the things that AI does is it makes interoperability matter much less. It doesn't matter if you have the same data standards. That's like a 20th century thing where we need this kind of file to talk to that kind of file. Where if you just have an AI that is like, can style transfer between different things, you can kind of create your own society that has its own sort of completely idiosyncratic mores and customs while still being interoperable with other societies because there's a kind of translation layer.

Yeah, it is interesting. We have the broader debate, like some people are really concerned that ChatGPT is shaded sort of left politically or denies certain truths or doesn't want to talk about certain things. Other people say, hey, who cares? What really matters is, is this thing a threat to us? And then other people will say, yeah, but I guess what matters is if people will use sort of justice concerns to justify sort of centralizing or regulating it significantly, then it really matters.

Yeah. I mean, but the issue is that the way at least in the US, politics is oriented is, and I'm guilty of this myself, is like borrowing clichés from 20th century politics, right? You know, every, every movement is the new civil rights movement. The environmental movement tries to structure itself that way. The gay rights movement, the LGBT stuff. It turns out that like even the, even the civil rights movement really, you know, there was a mass mobilization component to it, but, um, you know, all, all the, all these changes were, were were not just the result of getting a bunch of teenagers to rush Congress or something like that. And so I think this is happening again with AI where people are trying to plug it into this box of anti-discrimination and the kind of human rights orientation of these questions. And there are real questions. There's questions around how is your loan being approved? Is it using some opaque machine learning model that is Implicitly biased. And those are interesting questions. They're just questions we've been debating for the last 10 years and it's qualitatively different from building general systems that can fundamentally replace and transform human labor. Unfortunately, the discourse right now in DC is so in that former mode where everything is like— they're still debating about face recognition technology being insensitive to melanin. There are probably systems out there for which that's true, and especially in government systems where they don't have the most up-to-date technology. But as a technical problem, that was solved. We just gave it more data.

Do you have a mental model for when AI will make us better at our jobs versus when it will replace us? Is it like if you're an employer, it'll make you better, and if you're an employee, it'll replace you? I'm being glib, but how do we make sense of the impact on economic transformation?

So I think of this in terms of Baumol's cost disease. The right way to understand cost disease is really just differential productivity growth. When you have rapid productivity growth in manufacturing and the goods sectors, natural resources and extraction, digital services, IT, all that stuff is like, have very healthy productivity growth, but yet like We're still doing healthcare roughly the same way we have been for years. We're still doing education roughly the same way we have for years. There's sort of a, there's a need, there's a need to have the same number of doctors, the same number of educators, but their productivity hasn't actually increased. But to retain those workers, your wages have to continue rising anyway. And AI, I think, is going to do something very similar where there's going to be folks for whom it's incredibly productivity enhancing. In a way that's kind of like natural and intuitive. Like you already see this with programming, but there are going to be sectors where AI is, at least in the short run, kind of impervious to, it doesn't actually help with productivity. But because the rest of the economy is going to be like having a productivity explosion, there's going to be a need to sort of squeeze out more productivity from those cost disease sectors. And that might, may happen from like monitoring technology. Like, I think one of the things that AI is going to do really quickly in the short run, especially with Microsoft's stack, is make like manager monitoring tools on everyone's device where like, you know, it doesn't just look at your KPIs. It's like, or your, your simple evals. It's like tracking exactly what you did and coming up with like a kind of context-sensitive, like high judgment assessment of like. What you did that day and did it contribute to value?

Do you think that, like, something the EA people are sympathetic to is the idea that, hey, humans aren't the most advanced life form and we just need to let in whatever's the most advanced life form and that's going to be something after us and we will help usher it and we will be taking our place in the train of evolution or whatever that is. Whereas other people say, no, we are humanists and we can actually, or should strive to kind of control these things to some degree and not become extinct or fall. Where do you stand on that? Is that something that you think is going to be a real debate over the next few decades?

I mean, I'm a humanist because part of it is I don't think that you can step outside society and take a God's eye view or have a skyhook to Outside view morality. And so I think it has to come from how you're embodied. And it may be the case that there are like higher life forms. I think that's a bit of a fallacy. Like evolution isn't a gradual series of improvements. It's, we're not more evolved than our hominid ancestors. Like we're all, we're all roughly similarly well evolved. And adapted to our niche, right? And evolution, like even the dinosaurs, they're not, they weren't more or less evolved than us. They were adapted to their evolutionary niche and our kind of intelligence emerges from that evolutionary niche. And so I don't think there's actually a basis to say that there are like higher level life forms. And if anything, like, you know, you get this from like reading The Selfish Gene by Richard Dawkins. If it wasn't for wasteful evolutionary arms races, the optimal evolutionary outcome would just be a gray goo across Earth, right? But it's because parts of that gray goo were like, I'm going to get more sunlight by poking my head up. And then it cast its shade on the rest of the goo. And the other— so then there was a selection pressure to like— and then all of a sudden we have trees, right? And trees are only as tall as they are because they're in this constant arms race with each other for sunlight. And really everything amazing, everything beautiful in life comes from that kind of waste. Music, art, like that's all an indirect form of sexual competition. And so, you know, I worry that the real deep EAC people are, you know, they're like longing for the gray goo. And I don't really think that if we just covered the earth in TPUs, and let them max their entropy and pull all the negentropy from the sun. That's not a better world, right? In some ways, it's actually the opposite lesson. We don't just want to minimize negentropy. Actually, waste is good. Waste is where meaning comes from.

So you don't think it's a better world. And is it fair to say you also don't think it's the default for where where things are going pending any intervention?

On a long enough time horizon, I think it's plausible for great filter reasons. There is this remarkable sort of self-similarity in the universe where if you go down to the Planck scale and then the dark energy horizon, the full horizon of the observable universe, the geometric mean between those 2 distances is roughly human-level scale. Right. So we are like middle, I think, you know, Dawkins also coined this term, like middle creatures. We live in the middle scale of the universe. And for reasons of entropy, that's also where the most complexity is. Right. So we're kind of like on this knife edge of complexity between the smallest things and the biggest things. And our brains are like the most complex systems that we know of. And there's like this remarkable, like, similarity between the brain and other structures in the universe. Like, if you look at, um, you know, there's been all this work done on, uh, in interpretability research on the problem of superposition, that, uh, a single neuron in a neural network may be encoding multiple features. And so it's hard to interpret that feature. And the reason they call it superposition is it's like directly analogous to superposition in quantum mechanics. And if you think about a neural network, maybe the universe is in some ways describable as a convolutional neural network. And so the universe is rediscovering it in a self-similar way in our brain at the knife edge of complexity. And so I think they're like, I'm sympathetic to the EAx on that level where they're actually seeing these deeper time step things that are moving through the universe. And so maybe Maybe we're in this weird false vacuum and at some point things will tip over and we'll go full Matrix. I don't think that is anytime soon though.

Fascinating.

Shifting gears to something a bit more practical but also disorienting. Give us the right mental model for why government has become less functional. Is it much more nuanced than the idea of, hey, these institutions were set up almost 100 years ago, or some of them at least, and some of them even longer. And the idea that the incentives were just to continue to grow because that's how people, the bigger their budget, the more influence they had. And there really isn't accountability from the electorate or from a market. And many of them can't get fired, et cetera. So there's, it's just like structural or what's the right mental model to have?

I mean, there is a kind of baseline where institutions Again, for entropy, get more disordered over time. And I know entropy is about microstates versus macrostates and stuff like that, but one time I tweeted that bureaucracies have become less functional as an example of entropy. And people, the physicists on Twitter were very particular in pointing out that statistical mechanics doesn't have anything to do with institutions. But I think it does. I think there are, again, self-similarity between the big and small. And when you look at government, there is this general accretion. This technical debt is a term that Eric Weinstein has used. But then you think about why do human bodies, why do biological organisms, why do we not suffer from the similar kind of entropic forces? And obviously an individual organism does because we age and die. But the reason is because we have external sources of energy. We have the sun and we have food and we have sexual selection and mixtures of our genes together and so on and so forth. And so I think institutions, when they're effective, have to have similar sources of external energy inputted into them. And I hope I'm not being too econophysics with this, but that's sort of like, that's what FDR was, right? Like we had a system that was ill-adapted to the changing technological reality of the Industrial Revolution. And across not just the US but other countries, they had statesmen who kind of were able to see from one local minima to another and make the leap in a way that wasn't possible just through gradual gradient descent across institutions or something like that. And so I think the US institutions have sort of fallen into a similar kind of local Maxima or local minima. And the reason we're struggling to leap from one equilibrium to another is we've done everything imaginable to make that impossible, right? Not only do we have a polarized Congress that makes it hard to do major structural reform because of the lack of consensus, but we also have this sort of fetish for procedure where, you know, in the past, In the past, like, Vannevar Bush wrote a 14-page memo to FDR and got the NDRC approved, which led to the Manhattan Project in a few weeks. Today, that would be like a 10-year environmental review, and then you got to do stakeholder meetings and so on and so forth. And so it's like this proliferation of veto points. And it all connects in the sense that it's created a lock-in. And removed the normal escape valves, the normal mechanisms for moving out of that local minima.

Tell us what's misunderstood or what misconceptions people have about how influence really works in DC or how it's secured and maintained.

Yeah, so it's totally different than other countries as far as I can tell. So I work in a think tank, work in public policy, and As I said, I'm Canadian. There's no think tank sector really in Canada, and that's because the parties develop their own policy. And if they need expertise, they can go get that expertise, but you don't have this whole ecosystem. And the US is very different, especially beginning in the '60s and '70s with the decline of the strong party system, the new primary system, the party structures kind of got very weak. And labor federations began declining, church memberships began declining. And so all these membership organizations which had defined kind of the Tammany Hall kind of era politics got supplanted by this new thing, which after the war, the post-war growth led to very large wealth creation and the Henry Fords and so on. The Rockefellers and whatever, they created private foundations and those private foundations swelling on post-war growth seeded the creation of this nonprofit ecosystem where instead of going to the labor union and negotiating something with the Department of Labor or whatever, that still happens, but you also end up with a bunch of independent nonprofit think tanks, policy shops, advocacy networks, activist groups, legal aid societies, XYZ law centers, and it's been this kind of decentralization or diffusion of political power. And it's sort of the source of what we were just talking about with that sclerosis. In Canada, if you have a majority government, it's sometimes been called the benign dictatorship. In the late '90s, early 2000s, Jean Chrétien, with his majority, cut the federal government by 30%, fired Just straight up laid off like 50,000 civil servants and did it basically unilaterally. And there was a mechanism, there could be an election called if he went too far and he could be removed from office. But there's been this trade-off between, do we give a person a ton of power but make it easy to fire them if they cross a line? Or do we make it hard to fire them, but we strip them of all their real power? And the US has fallen into that.

In our last 10 minutes, I want to do a rapid-fire lightning round with you where I say a person and you let me know kind of your most interesting disagreement with them or difference in worldview, even though you may agree on many other things. First, let's say Balaji.

It's not an interesting disagreement, but I don't think Bitcoin's going to $1 million.

Yes. How about the GMU crew? You could pick Tyler, you could be anybody.

I think Robin Hanson is much more bearish on AI than I am, and I'm not quite sure why. Tyler, I think Tyler's views have evolved closer to mine over the years where he used to be more of a straightforward consequentialist and Benthamite kind of utilitarian, and now he's more of a pluralist. Bryan Caplan, I disagree with on a lot of issues. I'm not for open borders, for example, but I sort of respect them all because they To me, they kind of embody different, the different metaethical schools, right? Like Tyler's the pluralist. Robin Hanson's the brain-in-a-vat utilitarian. Bryan Caplan is like the Baptist deontologist who's like, it's wrong. It's just simply wrong not to be allowed to move between borders. So like we should just end borders, right? And then Alex Tabarrok is like the objectivist because he's the real Randian.

Is the best argument for pluralism really just the impossibility of being a utilitarian in terms of the ability to calculate utility effectively and that it's just unknowable and thus you should be a pluralist?

I think it goes back to an essay Charles Taylor, the Canadian philosopher, wrote called The Diversity of Goods. And it's his criticism of utilitarianism as elevating one supreme good above all others. And I think fundamentally there is no monopoly on the good. And I think the essence of pluralism isn't just pragmatic, it isn't just practical, but literally on a metaphysical level, there is no single definition of good. And there's kind of incommensurabilities between different people's conception of the good. And so liberal democracy, the essence of liberalism is to create space for those different conceptions.

That's compelling.

How about Ezra Klein?

Ezra Klein is far too sanguine about the Democratic Party. I think there's been some interesting parallels between Ezra Klein's thinking and some folks on the kind of new right that have gone understated. So for a while, a lot of what Ezra was doing was making libertarian arguments respectable to progressives. And you kind of see that with supply-side progressivism and so on. But if you read Why We're Polarized, his book on US polarization, If you read that alongside Chris Caldwell's book, The Age of Entitlement, they make the exact same arguments. And one is called reactionary and the other one is this insightful thing that using social science to learn about why we're polarized. But they both trace polarization to essentially the Civil Rights Act and the kind of reorientation of American political economy that followed. It's a very different difficult thing for him to say out loud. And a lot of his embrace of critique of proceduralism and so forth is again sort of translating some arguments that have been on the right vis-à-vis the professional managerial class and Burnham and all that. And I think ultimately he just puts too much stock in the Democrats being the main changemaker. Yeah.

Interesting. I read both those books and what I took from the Ezra book to simplify crudely is this idea that At one point, both parties were racist and thus we weren't polarized. And now only one party is racist and that's why we're polarized. I mean, I'm simplifying radically. Whereas Caldwell says we have 2 constitutions. One is the civil rights— one is the original constitution, then one is the civil rights law. And they're incompatible because one is about individual rights, freedom of association, et cetera. And one is about group rights. And those individual rights and group rights contradict each other and we don't have a way to reconcile them today.

Yeah, I think that's right.

Does that resonate with you? That Caldwell critique? Is that the fundamental tension in society between individual rights and group rights?

I think it's a big part of it. I think just on a practical level, it's dangerous to have 2 distinct understandings of 2 different civil religions, basically. You know, from my Canadian perspective, obviously everything is more fraught in the United States where race issues loom a lot larger. But, you know, in Canada, a lot of the conservative movement in Canada was sort of defined in opposition to like the creation of these human rights tribunals and, or like the Jordan Peterson stuff where like, you know, suddenly there's like new speech police and stuff like that. And because we were descended from the British system, Canada didn't really have the same kind of Bill of Rights written constitution the United States had. And that was much more flexible. That was much more pragmatic, sort of small-c Burkean conservative. You can think of the Charter of Rights and Freedoms in Canada, which was sort of like our Bill of Rights that came up in the '80s, as sort of like an example of Americanization. And a lot of Canadian conservatism is about resisting the cultural imperialism of our neighbor to the south, right? And so it's easier for me to say that this is like a big part of it because it was— it is this like deviation from a British classical liberal tradition, uh, that emphasizes rule of law, uh, you know, Hayek's norm of non-discrimination and generality, to something that is actually more, you know, has— comes from Valid places, like repairing the damage from slavery or something like that. But over time has become co-opted into new new forms of patronage, a new new sort of like it's sort of metastasized in a way that has challenged those those antecedents norms and constitutions. And it's the reason it's it's so fraught or so dangerous is because it. These are constitutional crisis-level questions and one of them has to win. And the Claremont people on the far right in American politics, they frame themselves around this as well. We just need to win and reset the Constitution. And then people on the far left are like the equity movement. A lot of that subtext is like, we need to be the party of the Civil Rights Act and impose this new Constitution. And really uproot the old system and that it makes politics more existential. And this is ultimately why I'm sort of skeptical of written constitutions overall.

Do you have a sense that one of them will win or a prediction?

One of them has won. The Civil Rights Act won.

The equity one?

Yeah. And I think there's, again, it's going to this basin of attraction. thing where it's very hard to roll that back and it's a path of least resistance.

Any last things to wrap up this conversation that we haven't gotten to but are top of mind for you?

It's been great. Thanks for having me on here.

Yeah, it's been a fascinating conversation. For people who want to go deeper, I highly recommend Sam's Substack and following him on Twitter. We'll have links in the show notes and Anything else you want to tease upcoming for folks?

I'm going to be working pretty exclusively now at FAI on AI and tech and institutional questions. My big project right now is trying to make the case for a kind of Manhattan Project for AI. And it piggybacks on a lot of what we've been discussing here, that we need to be proactive, that we need a new institutional mechanism that has a sort of great founder dynamic where it has the flexibility and nimbleness to reshape institutions. And that instead of trying to decelerate nuclear or atomic weaponry, we should be trying to master it. And we should also be trying to master AI. So that's where I'm spending my time right now.

That's awesome. We'll have to have you back on our AI show with Nathan Labenz and interview you on that when it's appropriate. Sam, thanks so much for coming on the podcast.

Thank you.

Is a show from Turpentine, the podcast network behind Moment of Zen and Cognitive Revolution. If you like the episode, please leave a review in the Apple Store. Turpentine is a network of podcasts, newsletters, and more covering tech, business, and culture, all from the perspective of industry insiders and experts. We're the network behind the show you're listening to right now. At Turpentine, we're building the first media outlet for tech people by tech people. We have a slate of hit shows across a range of topics and industries, from AI with Cognitive Revolution to Econ 102 with Noah Smith. Our other shows drive the conversation in tech with the most interesting thinkers, founders, and investors like Moment of Zen and my show Upstream. We're looking for industry-leading hosts and shows along with sponsors. If you think that might be you or your company, email me at erik@turpentine.co. That's erik@turpentine.co.