AI Is Changing Everything — but What Is Everything? A Conversation with Kevin Roose

Good afternoon. Good afternoon and welcome. It's a pleasure to welcome you to this inaugural event in our Dean's Lecture Series and to extend a special welcome to our alumni and friends at UMass Amherst who are joining us from around the world. I've been told that we may even have more people viewing the livestream than in the room today, which feels very on brand for a conversation about technology. Anyway, those remotely, welcome. I'd like to begin by thanking Donna and Rob Manning for their transformational support of the Manning College, including the Donna M. and Robert J. Manning Dean's Fund, which supports this series and so much more across the college. Thank you. We created the Dean's Lecture Series to provide a forum for bringing together perspectives from across the university to engage with some of the most pressing and complex questions in computing and society. At the Manning College, our work is grounded in our commitment to computing for the common good, the idea that the technologies we build should serve society in responsible and impactful ways. It's hard to imagine a topic more timely or more consequential than artificial intelligence. Its effects are shaping how we teach, how we learn, and how we live, and those effects are only accelerating. We're fortunate to have someone with us today who has been at the forefront of helping the public understand these changes. Kevin Roose is an award-winning technology columnist for the New York Times and the bestselling author of 3 books, including his most recent, Futureproof: A Guide to Navigating the Technology Future. By the way, I hear there's another book coming out soon. He's also the host of 2 New York Times podcasts, Hard Fork, a weekly conversation about the rapidly evolving tech landscape, and Rabbit Hole, an acclaimed series exploring how the internet shapes our beliefs and behaviors. His work examines the intersection of technology, business, and culture with a particular focus on artificial intelligence, social media, and the broader societal implications of emerging technologies. Ken will be joined in conversation by my colleague Ethan Zuckerman, whose work many of you already know. Ethan is a leading scholar on the social and cultural impact of technology and a key voice in the field of public interest technology, an area focused on ensuring that computing that computing serves the public good. Ethan is an associate professor of public policy, communication, and information here at UMass Amherst. His research explores how media and digital technology shape civic participation and public discourse, and he is the author of Rewire and Mistrust, 2 influential books on global media and institutional trust. Following their conversation, we'll have time for questions from the audience, and then I hope you'll join us for a reception downstairs immediately After the program, please join me in welcoming Kevin Roose and Ethan Zuckerman.

Keep thank you so much, and Kevin, thank you for taking a break from your vacation to be here with us. Just so that everyone knows, it'd be nice to tell them what you otherwise could have been doing today.

Well, I just left Disney World where I was with my family for the last couple of days. But honestly, and you know, please don't tell anyone in my family, it's a nice break from carting their four-year-old around Epcot.

Usually people get off this game and immediately say, "You know what I'm going to do next? I'm going to go to Disney World." But you, I prefer.

Yeah, I don't think they're going to reuse Kai Rispie Temper Tantrums today. Although, who knows? His cue was spicy.

Things couldn't get a little spicy. It's an interesting moment. We are we're nearing the end, but I've met a semester. It is week twelve of the thirteen weeks semester, which means that for a lot of us, the end of work is is sight. You had an interesting conversation recently with someone who told you that there are three hundred days of work left. What did he mean, and what does that mean for my job and for your job?

Yeah.

So I, I come to you through San Francisco. That's where I live. And in San Francisco, we have been gripped by this new religion. This new religion, as sometimes known as AGI, being AGI-pilled. But this is sort of the idea, uh, that we are on the cusp of a transformation unlike anything in U.S. history, probably at least since the Industrial Revolution, probably ever. And people who are AGI-pilled, Do all kinds of crazy things. They, uh, some of them sign up to have their corpses frozen after their deaths so that AI can reanimate them later. Uh-huh. I met one guy who doesn't wear sunscreen to the beach anymore because he's so convinced that AI is going to solve skin cancer by the time it's relevant to him. Uh, people don't invest in their 401s anymore. It's a crazy place right now. Uh, but one of the conversations I had recently was I wanted to get out of the house with a researcher. He said, uh, you know, I think you've got about 300 days left of work. And I think I just, I thought to myself, like, oh, this year?

Like, I'm doing the math.

And oh my God, I realized, oh no, he means ever. Uh, and, you know, I think what he meant was that in his field, software engineering, there are many people who predict, uh, that we are no longer going to have software engineers for some of these None that are recognizable do the tasks that they do today. I think what this person asked, and that was broader, I think it probably was something more like all white-collar knowledge work, which to be clear, I don't share that belief. So I was merely passing it along. It's sort of a little bit the crazy people in San Francisco belief, but this is very much their inside the bubble. This is the expectation, not 300 days necessarily, but the That we have sort of single-digit years left before large spots in the economy are transformed. Agents are doing all the work. Humans are there to kind of supervise old hands. Uh, and just the world as we know it will cease to exist. Not necessarily in a way that we all die, but maybe we just sort of become relegated to kind of like, uh, Nephilim or something like that.

No. You talk about, you know, coming from the AI bubble, from the land of the AGI cult. And I will say, you publicized this honestly. I discovered your work for the first time, uh, literally while you were an underground dissident, wrote, uh, a brilliant, and I think, uh, a book that people should be reading more often, uh, called The Unlike Disciple, uh, about essentially going undercover at Liberty University, uh, and just sporing, uh, The ethics and the ethics of that place, from the perspective of someone who'd be raised quite cyber. I think you may have gone a little bit further over. You know, now as someone who lives in Alabama, that's sort of working on this in real time. You had to fend off an hammerless search engine today by Microsoft, and a lot of you will be very right. You've gone fairly deep down the cloud code rabbit hole. Uh, what's your relationship to AI and to AGI? What are you using it for? What are you not using it for? What are your thoughts, perhaps?

So I use AI all the time. And this is very controversial. I'm not in the minority among my colleagues in journalism, but I find it enormously helpful. And I just finished, uh, well, just as of tomorrow, will I finish, turning in drafts today, uh, a book about the race to AGI and There is so much about the writing of any work, but especially something as long. Look, it's a book that is not about the actual words on the page, right? It's all the research, it's the fact-checking, it's the vetting and the sort of endnotes, the citations. And I have basically offloaded everything that I'm able to offload ethically and professionally to AOPs. So I have a team of Claude that I write with. Obviously I have human editors, I have a human researcher as well, who are wonderful. But for things like research, where I can just now put all of my materials, all of the articles, all of the interview transcripts, all of my source material into, I use a couple different AI tools, but into NotebookLM in this case. And I can just ask it questions. Who are the 3 people in this meeting in 2019? Or what's the best argument against this line of technical research? What might this person have been thinking when they said this thing? It's been enormously valuable just for me to have a sort of living database of all my source material. Uh, and then I have a team of agents who review, uh, my work as I, as I feel like pretty good about it. I have a fact checker that goes through chapter by chapter. Uh, in my first break, it was hard considering an AI, because a couple of years ago, it was making stuff up all the time. And we were having to fact-check it. It was fact-checking us. It catches some things that are actually mistakes that I've made that I can then go back and fix. I obviously am also working with human fact-checkers. So all of this is kind of experimental and new, but I found it enormously useful. I think it's probably saved me something like a couple years of research on this. But I've established 2 bright line rules for my own use that I had to talk about this in the foreword to the book, which is one, The words of a head jury, like I am not having this write for me. Uh, both because I think that would be, uh, ethical conundrum and something that I, I am not allowed to do when there are times, but also because it's just not that good. Capture into subtleties of voice, it produces stuff that sort of looks good and coherent, but you dig in like one layer, it's like some of the ideas just don't connect. Uh, and there's a second rule other than sort of I write my own words. was that I'm just not feeding any sort of sensitive documents into a cloud-based AI system. So if I have a confidential memo that's been sent to me, if I have something that's getting leaked somewhere, I keep that pretty well fenced off from the errors.

So, so how do you navigate the gate between your friend who believes that there's 300 days of work left, really irascible about it, but Clearly part of the vibe, the type of world that you're coming from. Uh, and very real, uh, you don't use AI, right? Not just games, it's in app— well, can I dream? Because it's just not that good at it yet. Uh, it seems like we're going through this revolutionary moment where AI has done very, very dumb things. Uh, I think almost anyone who has coded in a serious way Is in that way to sort of concede that a lot of that job, you know, that a lot of that job can be as well done by someone that copies codes as it can be by humans? It may be possible. There are other fields, perhaps not the perhaps very right fields, that are going to fall to the same strategies of scale, to the same strategies Particularly forms of reinforcement learning. We are at a world-renowned university. We've got people studying everything from nursing to turf science to philosophy to a lot of computer scientists here. How many of these fields do you think are going to get transformed? Do you see a moment at which AI becomes good enough to be replacing your writing? Or really feel, do we think of this transformation as just an ironclad?

That's a great question. There's this concept that people in Silicon Valley talk about, about jagged intelligence, because it was a term that was sort of popularized by a researcher named Iray Farhati. And what he meant by that is essentially what you just said, Dorothy Spears, like deep spikes in AI capabilities. So for example, AI is now, you know, in coding competitions, it is better than almost every human coder at those rare competitions. It is outdoing many math PhDs and physics PhDs in their chosen fields. It is doing incredible things with genetic coding search systems that can go out and work autonomously for hours at a time. But if you give them simple— they sometimes fail at things that would be relatively simple for humans. So what we had is not sort of a perfect representation of all human intelligence. It did not all arrive at once like some of the early AGI people did. They thought we would have this sort of burst of capabilities explosion that would sort of rise the whole water level with progress. And I think what we're seeing now is that the gains in some fields are just much bigger than others. So coding is For this long. I think that's what people mean when they say 300 days left of work. That is the feel that they're talking about. So I think in some sense, that's a technical problem. It's just easier to do, to build these systems to be quite good in domains where there are like, we are writing wrong answers too quick. You can train a model to go out and write a bunch of code and then write unit tests to see whether the code works. And you can do that over and over again using this technique called reinforcement learning. Until the model has stopped making errors, that's not possible to do in every domain, and not every domain has clear right or wrong answers. I think you know that in a funny way.

This is actually kind of a little trick. We were talking before about computers learning how to play checks, and you know we ended up quite a while with Deep Blue facing up against Garry Kasparov, and and. That's a hit eventually. Uh, that's a game where you can literally look at it and say, yes, you won. Good job. Pat on the back. Coding feels like something where we often give the pat on the back. Why is writing an algorithm, well, if the output from this—

Oh, it's a more subjective, and I think it's also connected to this concept of self and expression. Uh, when I use an app on my phone. I don't think that like the code that is running in the background is an expression of the programmer's authentic self. Like, it is a functional thing. Um, and coders get very, you know, sometimes they get very upset. They think they, you know, they do express themselves. But I think for like, for most users, uh, they, they just care whether the thing works. And I think writing is not like that. We've seen now in, uh, actually romance novels are sort of the first Genre of fiction that has fallen to AI, or is that it, that is the process of falling to AI. Uh, and romance novelists are out there churning out dozens of books a year using AI, but they're not telling people for the most part that they're doing this because they found that the moment you say this is AI-generated or even partially AI-generated, uh, nobody wants to read it. So there's something about at least the illusion we are hearing from a human. Uh, and, and the inner thoughts and experiences of a human that is somehow rooted in, in personhood, uh, that makes us, uh, believe in it and value it more, uh, than we would if we just thought it was a bunch of, uh, AI. Even if the text generated by the AI is better at some, uh, sort of discrete level, uh, we would still want that to come from a human because part of what we're doing when we read a book is trying to connect with the person.

This is probably the moment at which we should make our, uh, disclosures. Uh, I, I've decided to go full Elon on this. I've got to pay for a Petzl. Uh, you got the headset, uh, with Anthropic's new Mythos, uh, going directly into your head. This was what you renegotiated.

It's in my neural ICU plant. Yeah, I got, uh, 17 models running inside there.

So your last book before this new book about AGI was called Creature Perf, and it was this idea, and, and this was a while ago, back in, uh, 2021. Uh, like Wolves in the Age of Automation, you were telling people at that moment about this idea of sort of leaning into their humanity. And you, I think, just made a fairly effective— you need to understand that I am a real human making the judgments about the words on page. I am putting my words into the podcast. How was that applied more broadly to the folks in the audience here, particularly the folks in the audience here who do to one extent or another write code, whether it's a form of expression, whether it's a form of getting work done, whether it's the job that they have right now. I hope we might have freedom days for a while.

Yeah, I think interpersonal skills are about to become much more important in computer science for reasons that are actually like, I'm not predicting. But I was telling your faculty earlier, I was at a startup recently and you walk into their office and everyone's got, you know, their 4 monitors set up. And then everyone's also got like a little microphone at their desk because these programmers, they no longer code with their fingers. They code by talking. They just tell their agents, hey dude, solve this problem or adjust this thing or run this test or build this RL environment. And the agents just go off and do that. And it sounds like a call center, uh, but what they're doing is communication. They're not communicating with each other and they still are communicating with each other. More of their time when they're actively working is spent communicating with these agents and they're finding that actually the skills that make you good at communicating with humans, you also need to use them with, with these AI systems because you're going to get better work out of that if you give them clear instructions. Uh, clear metrics for success or failure. If you come to them in a, in an empathetic way, for whatever reason, it seems to produce better code. So there's an interesting kind of scenario unfolding in computer science, at least among the tech startups in the Bay Area, which is like people are now starting to learn empathy before the naysayers.

And it's becoming— And I admire it all translated down to the—

here we have 6 No, it's true. They're trying to not burn them out. Obviously, agents don't get burned out in the same way as humans. They don't like need vacations, but they do sort of— the quality of their work degrades if you give them a task that is too much for them to handle in a single session. So there's this interesting phenomenon where now every engineer is kind of a manager, and they're realizing, okay, oh, we actually have to teach this new kind of management to people, even if they work sharp. Uh, or only an individual contributor.

So interestingly enough, there's a direct parallel to this. Uh, being a professor at, at Harvard, at a, at a university where a big chunk of what we do is career research, uh, we have a very powerful flexible arrangements. They're called grad students. Uh, they are, uh, a bit of a, a, a scarce life force these days, uh, having a lot to do, uh, with funding changes in the environment and such. But this is a conversation that I know a lot of fellow professors are having, which is, I can hand this task to a grad student and I can help them learn how to become a researcher. I can help them learn how to formulate problems and so forth. I'm out there. Or I can write a block of code and I can probably get the results faster and I don't have to hear about the fact that they got logged out of their apartment. Uh, this is a real moral challenge for the world of research. And the broader challenge is if it is getting easier, easier, easier to do work on your own without some magic team of humans, how people learn, how would be the— How do you learn to ask those research questions? How do you learn to do that really detailed sort of work? Um, that I'm guessing you and I would agree probably still is going to require a human touch, at least for some time.

Yeah.

I mean, what we do in, in a sort of fully human setting is we just have feedback, right? We have training, we have feedback sessions, we have peer reviews, we have performance reviews. Um, that happens once a year at a lot of places. What if you could do that every day with your agents or twice a day? Say, hey, how am I doing at giving you the kinds of instructions that are producing good work. And obviously they're going to flatter you if you don't instruct them to. They're going to say, oh, you're the best manager we've ever had. You're so amazing. I'm so lucky to be working for such an empathetic genius. But with some skillful prompting, you can actually get them. I mean, one of the things that I love to do with my agents when I'm writing is I have one agent write a really positive review of whatever I've just written. And one write a completely scathing takedown. And I then have sort of an agent who synthesizes and says, well, this point looks a little more on the nose. And I find this very helpful. So it has just tightened my feedback loop a lot. But I think you're right. There's some piece of learning to be a good colleague, a good manager, a good student, a good teacher, That is just about showing up and putting in the work. And I don't know that you can fully replicate that with agents. So I think there is still a valuable role in training people how to give feedback, how to receive feedback, how to work on a team, how to express leadership. Like, I think those are all very, very valuable parts of what students should be learning.

We, um, we were talking earlier about one of the problems in academia at this moment where when you're studying AIs, The academic publishing cycle is so long that when new research comes out, it may actually be out of date. So I'm almost certainly going to cite some out-of-date research here, but there is research that suggests looking at software that's been put into public repositories, things like GitHub, researchers have gone and sort of looked to see Who seems to be making the best use of these models? And what the researchers found in that particular study were that people who were experienced programmers, people who really knew how to code in and out, seemed to have a real productivity increase using these tools. People who were much newer to the game were starting to use these things, didn't seem to be using them very well, um, looked like they were actually having a skill reduction. in using some of these things. Now you may be ready to tell me that the new tools are so much better than this, that isn't going to happen anymore. The question I'm trying to cue up is that we have a real challenge as educators helping people become computational thinkers. How do we get people to think in terms of how a machine solves a problem? Historically, we have done this by asking people to do the very painful task of coding a computer for hours and hours on end until it literally reshapes their brains. Now you can get your way through almost any intro coding class by feeding your assignment to Claude Code, and it's probably going to do okay. Uh, and unless we do some of the things I've seen some of my colleagues do, like have people do blue book coding exams, it's hard to know whether we're going to be able to teach that sort of computational thinking. Is there hope for us around that, or should we assume that these tools are getting to the point where having that sort of knowledge may not be as important as I think it is?

I don't know. I'm not a computer scientist, and I always try to express my own doubt about my intuitions when it comes to fields that I'm not an expert in. My impression is that the What is the changing in the changes in coding are just sort of moving several layers up in abstraction, right? So we used to teach people assembly, right? How to how to like code like literally into the computer, and then we abstracted that to programming languages, and those programming languages got more abstract over time. And then we had you know object-oriented programming, and like all these changes in in computer science have basically been moving from like something that talked directly to the computer to something that sort of talked. to a set of abstractions that then talk to the computer. And so I think on one level, what's happening with AI right now is just like another layer of abstraction. It's like you still need to understand something about what is going on. But for example, I don't code and I've built probably 20 apps in the last 6 months. And parts of it would, like, I could have saved myself from time or some time if I had known something about coding. But other parts I think I was probably actually more effective at than someone who's a professional coder because I was just coming in completely blind and I turned sort of the planning and thought process behind the code over to the coding assistant. And so I was not coming in with my own preconceived notions of like, oh, we're going to need a SQL database for this and you're going to want to write it in this language because this language is better for this kind of thing. I was coming in as like a total neophyte just saying like, hey, I want an app that does like these 6 things. Can you kind of figure it out for me? And it would do that. And so I have learned some computational thinking just by looking at what the model is doing on my behalf.

And you can actually now expect the so-called chains of thought, which I find fascinating. This is my favorite part of using AI right now is that you can actually see the inner monologues of the AI systems. And as they're sort of trying different things, oh, that didn't really work. Let me back out. Let me try it in a different language.

I find that I'm learning a lot from just watching that work. So is that useful, applicable knowledge?

Maybe not, but I think it is teaching a kind of computational thinking that is more attuned to the realities of tools.

And I think we're super unique in this. We have at least 2 categories of people, possibly 3 categories of people. Category 1, Um, are the, the professors and the faculty who are often trying to solve giant research problems. And the world that you're describing in which you can throw a lot of documents into Google Notebook or another RAG-based system and be able to sort of query them, that's pretty exciting. Uh, it's pretty sexy. There are folks like, uh, Justin Gross sitting in the front row there who I know are trying to think about How data scientists do incredibly sophisticated work, both with and without AI. There's also a lot of people in this audience who are wondering whether they're ever going to be able to get a job and don't have tenure. And they are looking at the fact that Meta just laid off 8,000 people. They closed 6,000 open jobs at the same time. Microsoft is offering early retirement to about 9,000 people. So those folks are very interested in what world are they going out into. We also have in the front row some people who are really interested in how you keep a university open if the people who wanted to come here to get jobs now start wondering if those jobs are going to continue to exist. So can you make categories 2 and 3 happy, or at least offer them some sort of consolation?

Yeah, I mean, look, I am very glad that I do not have to solve these problems for a living. I have great admiration for the university officials and administrators and faculty whose job it is to navigate all this. What I can tell you from my perspective is that I think that there are a lot of institutions that are right now trying to kind of adapt the old ways to the new reality. And I think that that's going to be a hard thing to do because these models are getting so much better every single year that you are in essence trying to lesson plan a curriculum plan and plan the structure of a university around something that is going to be obsolete or on its way to obsolete by the time that you finished. Take it from me, I wrote a book about AI the year before ChatGPT came out. It is very hard to make predictions about the future when you're on an exponential like we are with AI right now. I'll tell you what I would do if I were starting a school. And obviously this is, I'm glad that I'm not starting a school. You should never send your children to my school. I would devote about half the days to just pure offline learning, like pure sort of Socratic, like let's read a bunch of books and let's talk about them. I would have no devices anywhere near it because I think there is, To me, the danger of sort of trying to hybridize everything is that you end up with the worst of both worlds. You end up with like the technology that is disrupting all of the work that's happening in the classroom and you're also not learning anything. So I would like half the days or maybe half of each day would just be like education as it existed in the 17th century. And then the other half would be like giving everyone unlimited budgets for these AI tools and just having them build stuff.

Yeah.

Because I think there is really no better way to learn about the technology than by using it. I have found my own learning has come mainly through doing. And so that would be the other half of the day is like, let's like stay pretty close to the frontier, like use the best tools available, figure out what they can do because this right now in the job market, these are the categories of people who are getting jobs, right? It's like the people who have skills, that have nothing to do with programming computers. There are still lots of jobs for them. And then within computer science and programming, it's the people who are really skilled at getting the most out of these tools. They're now, they're no longer 10, you remember, might remember the phrase 10x engineer. This was sort of the old term of art in Silicon Valley for like someone who gets 10 times as much work done as the next person. That's no longer impressive to them. Now they're talking about 100x engineers. And some of those are senior people who have gotten really used to the tools, but some of them are like 17-year-olds who are just like really, really obsessed with these tools and are using them every day and are figuring out how to like do amazing things with them. So I think that would sort of be the way that I would push education is sort of away from this kind of messy middle and toward these more sort of extreme sounding ends of technological adoption.

I I think you may have just updated Maria Montessori for the 21st century, at least for the Bay Area. So if you get sick of this New York Times gig, we can probably start up a school for AI-driven engineers.

My kid goes to a Montessori preschool and all preschool teachers should make $1 million a year. That is my actual view. That is a thankless and hard job.

We, we have a department of edu— we have an education department here who I think would be very, very happy to hear that. When, when we were talking before, you actually sort of broke down a distinction that, that sounds a little bit like how you were talking about breaking down that day. There is a set of jobs that are going to be automated. There's a really interesting philosophical question about how broad or how narrow That set of jobs is. My friend David Jensen was making the argument earlier today that it's a narrower set than you think, and that the hype people are very good at it. There's other folks who say, no, if we got enough data, we should be able to do it. And that might mean biology and then law and so on and so forth. But you were also acknowledging a human set of jobs where you think we're going to be resistant to automation or automation's going to be really hard to do. Again, keep in mind you're talking to educators. Keep in mind you're talking to people who genuinely are doing this for love since most of us could go work for Anthropic for more money. How do we think about educating people on sort of both sides of that equation?

Yeah, I think it's the right question. I think the thing that I have been trying to grapple with is sort of the difference between jobs and tasks, because there are occupations where we could have automated them a long time ago, right? Like everyone's got a coffee maker in their house. That is essentially a form of like labor automation. You know, a person used to have to make the coffee, now a machine does it. But we still have coffee shops. Why is that? It's because maybe the coffee is not the thing that you are paying for. Maybe it's the experience. Maybe it's a quiet place to get some work done or a nice place to have a conversation. Maybe you just like when the barista says, have a nice day to you in the morning. There are these little coffee kiosks. Anyone been to the San Francisco airport and seen the coffee robots? So they've got in every terminal now, they've got these robots that'll sort of descend with the giant claw hand and like make your latte. And, and they're not popular. Like they have had trouble keeping them open because even though they are faster and maybe even a little cheaper than the Starbucks that's like just, you know, 10 feet away, like people really want that human connection. There's something about that that feels good to us. And so I think there will be jobs where even after all of the tasks inside the job or a large fraction of them have been sort of eclipsed by AI. Like, we will still want that. I was giving the example earlier, like a therapist, like it will probably be true that shortly AIs will be better therapists than humans along kind of any conventional metric that you would use to evaluate them. They're a lot cheaper. They're always available. They've trained in every modality because they've been trained on all of the data on the internet. So you could make a case that the therapist is doomed, but I think there's a subset of people and probably a large subset of people who will just never want to go to an AI therapist. It just, even if the advice they're getting would be identical to or superior to the advice a human therapist would give them, that just feels bloodless. It feels bad. We don't like it. There's something about us that is hardwired to love human connection. So I think that there are one place where these sort of AI bubble often gets it wrong, I think, is insisting that as soon as a job can be automated, it will be automated because they see that for themselves and their companies. But I think that is not true in the economy as a whole.

I'm going to open this up in just a moment to the audience, and I'm going to roam around with a mic so that the rest of you can have some of the conversation that I'm having with Kevin. But I want to give you sort of a 3-parter. And it's sort of 3 versions of the same question. Obviously, you're writing your next book on artificial general intelligence. You've been following this as closely as anyone else. As you think about this transformation that we're in the midst of, what excites you the most about it? What scares you the most about it? But here's the hard one. What do you think we're missing? Um, you have written about an incredible range of subjects. Um, you are a much better technology columnist than I am, but we both have the challenge of trying to figure out what to write about in tech. And I've discovered that 85% of the columns that I write these days are about AI. What aren't we seeing? What aren't we writing about? So what excites you? What scares you? What are we missing?

What excites me is that I think if we manage to solve all of the big problems around AI, obviously there are tons of big problems, everything from addictive chatbots to environmental harms to just the existential risk that all this could kill us. Like, I think this could be really good if we get all of that right.

I was just, At Disney World.

And I went to, they have this amazing ride called Carousel of Progress. Anyone ever been on this? This is like an old-fashioned ride where you go in a theater and it sort of rotates around and they show you this sort of series of animatronic tableaus of like life in 1900 and life in 1920 and life in 1940. And like each time they have a little bit more technology. You know, at one point they have an icebox and then the icebox is a refrigerator and then they have electricity and they're not like, you know, doing the laundry by hand. It only takes them, you know, an And it's like, oh my God, life has changed. I watched that and I thought, well, we need something like that for today. We need someone to give us a positive vision of how our daily lives could be better 10 or 15 years from now. Because I do think that technology generally improves our life. I am not one of these people who like wants to go back to an agrarian economy. Like, I like my, I like my, you know, vaccines and my iPad. And so I do think that on the whole, if we manage to solve all of the contingencies, this could be really good for us. It could free us up to do the things that we love. We could cure a bunch of diseases. I'm very excited about what's happening in AI and medicine. My own father died of a rare form of cancer that I think could have been treated with better interventions. So I'm very excited about the potential for AI to help in medicine. That was the positive side. But what worries me is that we are just not ready for this. Our institutions, our government, we are not making the kinds of changes that we would need to make if all of the things that the people in San Francisco are saying about AI are true. I worry about existential risk. These systems are behaving in ways that their creators did not anticipate and do not know how to control. Some of you may have seen the recent coverage of Claude Mythos, which is Anthropic's new model, which is finding zero-day security vulnerabilities in 27-year-old software and appears to be wreaking all manner of havoc. And so they're releasing it to defensive cybersecurity teams before they're releasing it to the public. I think that stuff is very underappreciated. I think biorisk is very underappreciated, and I just don't think our government has a good track record of regulating new technologies. You know, we still haven't regulated social media, which is 25 years old at this point. So I think we just need to like get our minds around the pace of change here and really like start preparing for continued change. And then I think the thing that, I don't know, we collectively are missing is that The sort of old adage that like today's AI is the worst it will ever be. Like, I think we have just started to get our minds around collectively the progress that has been made in the last couple of years. But there is, to me, there is no obvious reason that these things will plateau at anywhere like at sort of around human level on a bunch of different tasks. And so I think the thing that is underappreciated is like, what if the weirdos in the bubble in San Francisco are right and this thing just keeps scaling forever and ever and ever.

That for me is perhaps the scare as well as the underappreciated one. But I think that notion of taking the weirdo seriously is an interesting provocation and maybe an interesting provocation in a room where I think you might have more skepticism than that. I'll be really curious to see where the questions end up going as far as sort of enthusiasm or skepticism. We're going to start circulating a mic in the audience. I just, because we are at an academic institution, I just want to offer a brief reminder about questions. Questions are interrogatives. They often end with a slight uptick in tone, indicating that you are interested in what Kevin might say in response to this. They are not in fact short speeches. If you find yourself saying, I guess that's not really a question, it's more of a statement, that is not in fact a question. And so if you're planning on heading that direction, please don't. I was actually going to circulate the microphone because I have a tendency to loom over people when they start giving speeches instead of questions, but I will stay seated and hope that my speech will do it instead. I would love a show of hands. of people who have questions, uh, that Kevin is capable of answering. And we can go down here and—

That was very good agent prompting. You're very good at this.

I, I, I, maybe I will actually have the ability to, uh, yeah, uh, uh, audience management is very much a form of agentic prompting. And, and please introduce yourself and, and tell us how you fit into the, uh, the UMass or the Amherst community.

Um, hello, my name is Sam. I'm a Queasy undergraduate. I study resource economics and will potentially graduate with like a data analysis certificate. So I'm sure you've seen the recent Allbirds move from a shoe company to like a pure sort of lugubrious AI holding, whatever. And I was thinking about what you said about Starbucks and about coffee and how there's always going to be a human component. But Starbucks in the past few years has been hemorrhaging money on the coffee side and is sort of in the process of becoming a bank through their, you know, like Starbucks points. I guess without, you know, going too far into detail, what is the, what prevents companies that are operating under like rational assumptions and economics from turning into lugubrious AI holding firms or banks or abstract financial firm operators? What actually holds those companies to continuing the production consumption loops? Why would you do that if your return is just lower through having to actually produce and consume things and being constrained by supply chains?

I love the word lugubrious. So thank you for introducing that into the conversation and the Allbirds thing that he's referring to. So there was this company named Allbirds that made these hideous shoes that everyone in technology wore for several years, and then they fell on hard times and got acquired at a massive discount and now are pivoting to becoming a cloud AI provider. There's going to be a lot more stupidity like that, by the way, because Sort of like, you remember a couple years ago when like companies would just say blockchain at the end of their name and then their stock price would go up 400%? Like that is happening now in the economy. I don't have high hopes for the Allbird computing cluster, but I think you're right that there will be companies that sort of just decide that it's, they, their model no longer makes sense, whether it's too labor dependent. Whether it's not, it doesn't have the kind of margins that their competitors do. I kind of think that this is going to come more from, like, less from the sort of established companies, which I think are having a very hard time sort of getting their heads around AI and what it means for them. But I think it's like true that in every sector, there'll be like a company with like 3 to 5 employees that tries to take on the big incumbent in whatever category. So there'll be like a 3-person insurance company that that wants to take on State Farm and they'll like go off with a bunch of agents and do that. There'll probably be like a 3-person newspaper that tries to like take on the New York Times and they'll, you know, make a run at that. Like, I think there will be businesses in all these verticals that just attempt to not to provide the same service with a much different ratio of human labor to AI labor. And for some that may work. And that may mean that you actually have to like pay a premium for, if you want the human therapist, maybe that cost goes up in a world with cheap and abundant AI therapy because that's sort of the premium experience that people want. And so that's one of the things I worry about is a new kind of inequality, not necessarily of wealth, although it will track to that, but like of access to human labor and access, you know, it may be that The students from disadvantaged backgrounds go to schools where all the instruction is done by AI, and the students who can afford to pay for higher-price educations will get the contact with the brilliant human professors that you all enjoy here. And then that will become sort of a luxury good. So that's one of my sort of fears about this is we're going to see a stratification.

I think Justin had a hand up here.

Thanks for the shout-out.

I was reminded, I had to double-check that it wasn't written by you, it wasn't, but an Upshot article from a couple years ago in the New York Times that stayed with me. The title was When AI's Output Is a Threat to AI Itself, and it was dealing with the fact, and as you were pointing out, maybe things have changed enough and it's not as much an issue in what we're thinking about. But part of the point was AI and thinking about whether it's going to scale and scale and scale is that humans in the loop in various tasks are providing variation and diversity that LLMs utilize in providing output. And if in lots of fields, let's say, Oh, we don't have any like, you know, workers or people who are in the loop who know anything about what's really going on. Let's just say it's about coding. Like, is it true that if no one's actually doing coding for themselves, I'm, you know, that we won't have this degradation of actually quality because everything is sort of going to the median output and there's not this, you can't really like control the temperature on the, on the AI. I'm wondering whether. I mean, maybe, you know, whether folks are concerned about what happens if humans are left out of the loop. Do we actually then get worse and no one knows how to do the thing when we have to go back to them?

Yeah, it's a great question.

This was a couple years ago, as I remember, and it was around this idea of model collapse or mode collapse. This was an issue that people were a lot more worried about a couple years ago because at that point, the way that you got better models was you just Dumped more data into them, had more compute, trained them for longer. Sort of that 3-ingredient recipe was how you got from GPT-1 to 2 to 3 to 4. In the past couple of years, most of the capabilities gains have not come from dumping more data into models. They've already exhausted all of the data on the internet. So they are doing some of this synthetic generation, synthetic data generation, but more of the gains are coming from What's known as reinforcement learning, or from having a model that has already ingested all of the data on the internet and turned it into parameters. And then it goes out and it attempts a bunch of tasks. It tries a coding task and it tries it 100 different ways or a million different ways and reports back, hey, this way worked. Maybe I can use that same sequence of steps to take on different tasks. So it's less about the data that's going into the model at the pre-training stage, which is sort of the first stage of training. And it's more about what tasks are you training it on after it has already ingested all of the data. So I would say that's not as much of an issue among the AI companies that I speak to. There is an interesting version of this though that I think is starting to become true, which is that people are really worried about the sort of novel human-generated writing that's on the internet going away and how that may make it harder for large language models to produce something that we would identify as good or human-like writing because so much of, I mean, by some studies, the amount of human-generated text on the internet has already been surpassed by the amount of AI-generated text. So it's interesting, like there was this concept around pre-war steel, which was like basically before World War II, there was steel that had never been sort of contaminated by radiation. And after the war, you had this like sort of subtly weaker steel because it had all been contaminated by background radiation. And so like pre-war steel was sort of this vaunted valuable commodity because it had never been exposed to the kind of background radiation after World War II. So I think there's something interesting with like pre-war like text, like before the bots took over all content creation, we had this like pristine internet that's like, that was filled with stuff that humans had written and maybe that's going to be sort of the equivalent of pre-war steel and like All of our Reddit threads from like, you know, 2010 are going to be seen as like very valuable.

As, as a child of the, uh, well, as a creator of the 1990s, uh, internet, there would, nothing would make me happier than discovering that our old Tripod and GeoCities pages from the 1990s turn out to be pre-war steel. Um, let's go and get a couple more, uh, questions in here.

What's that?

Um, I, I would, I would love some gender diversity. Do we have, uh, any, any female approaching people, uh, appearing people? We do. In the back. We will go back to the many boys with hands up shortly after. Just trying to balance it out a bit.

Hi, thank you so much. My name is Siobhan May. I'm a lecturer in the College of Information and Computer Sciences. I wanted to, I teach writing in the college and I wanted to come back to what you were talking about with writing and how the sort of human writing being so closely related to a sort of unique human expression and that making it, not that it's not sort of automatable, but special. I think you made the distinction between writing and coding. And so I was just wondering, you know, there's all sorts of different types of writing. What writing is automatable in your mind and what writing is not?

I think a lot of writing that is sort of functional is automatable. So product reviews, sports game recaps, stock market reporting, things that where the goal is to just convey information, your press releases, emails. Most writing is that kind of writing just on a sort of words per day basis. It's like, I'm trying to communicate something to someone. It doesn't matter if it's the most beautiful literary prose. But I think literary fiction is going to be very hard to automate because of this sort of quality of like, as soon as we know it's AI, we're not interested in it. Like, I have no interest in reading anything that a chatbot has written. And that's, I don't think, I think a lot of people share that feeling. So I think if AI makes inroads into literary fiction, it will be through authors just deceiving their readers and not telling them that they're using AI, because I think the illusion is an essential part of the value. So I think literary fiction, safer. Poetry, safer, same reason. Technical writing, business writing, legal writing, all of these things where like you're trying to get something into like a disciplinary convention. That is all very replaceable, I think. But I should say one interesting thing is that I think writing in those fields will become weirder because people will want to differentiate what they are writing from the things that the AI models are doing. And I'm already starting to see this now with like, I talked to an email marketer the other day, like somebody just sends out spam on behalf of like big corporations. And they said that when they put typos in their emails on purpose, it like increases their open rates by like 40%.

Have you seen the Angines de Poitrine phenomenon?

No, what's that?

This is a Québécois band. They are a pair. They are dressed head to foot in polka dots, including polka dots on their skin. They wear papier-mâché hats and they play microtonal rock music. And there have been 10 columns in the last month essentially saying, this is what AI can't do. And it's just literally people sort of looking to sort of say, well, thank God, at least we can still be weird. And I'm very interested to see, you know, when they—

I think this is genius. I think we should absolutely all be doing like our versions of this weirdness thing. I feel it in my own writing. Like I find myself trying to write weirder because I don't want anything that I Right, to be read as the outputs of an AI model. Like, I think that's very smart.

Let's see hands up for a weird question.

Weirder the better.

Great.

We've got a weird question over here.

All right.

I have a kind of a 2-part question. The second part is maybe the weirder one. Um, first part is, do you, like, what's your AGI timeline? Do you think that's even possible? Do you think it's possible with exclusively LLMs? And then the weird part, um, what's your P-doom? How much, like, I know, silly question, but how, how much merit do you attribute to the different, like, potential scenarios in the discourse, whether it be like biorisk or like the San Francisco sci-fi Yudkowsky, like, fuming stuff? Like, yeah, what's your—

I, I, that's not weird enough, but I'm gonna make it weirder, which is, so I, I want your existential risk answer. I want your P-doom answer, and I want your P-weird.

answer.

Do you think there is a chance that we're going to end up with systems that are genuinely weird and surprising?

Do we need to do like a calibration of like, who knows what P-doom is? That's like your probability.

It would be helpful.

Okay. So in San Francisco, the people all talk about P-doom. That's your probability of doom. So like the probability that AI kills us all or leaves us fundamentally disempowered. And then the timelines to AGI. AGI is artificial general intelligence. That's You know, everyone has a different definition, but most people, it's sort of like systems that are, you know, sort of at or above human level at most things. I think AGI is probably already here. I don't think we, I don't think we have, like, I don't think it's a very useful term because some people think AGI, for it to be AGI, it needs to be better than everyone at everything. And I think some, for some, it's like, it just needs to be like pretty good and capable and like something that you could delegate real work to. I tend to have a lower bar. I know like Demis Hassabis at DeepMind has a crazy high bar. He thinks it should be able to like discover Einsteinian, you know, relativity on its own. So yeah, I don't know that it like debating timelines, but I think we will have very capable systems. We already do. I think they will become more capable. I would be shocked if a year or two from now we didn't have systems that sort of met everyone's, everyone's like pre-2020 definition of AGI. I think that if you went back in a time machine, And gave your laptop to like anyone in 2016 who was working in AI and said like, showed them like Claude or ChatGPT and said, is this AGI? I think they would say like, yes, absolutely. My P-doom is sort of around 5 to 10%, which is a cop-out, I think, because that's not like, this is just a vibe, right? It's not a real, like I haven't, arrived at this through any statistical means, but I think it's lower than some of the biggest doomers in Silicon Valley and probably higher than most normal people. I think there's a chance that this all goes quite badly, but I think that the thing that's making me more optimistic is that more people are paying attention now. It was just a couple of years ago, I was writing about AI and how these systems were getting quite good. And like people didn't want to hear it. It was like, it was just too abstract. They hadn't seen the tools. They hadn't played around with them. And now I think people really understand that there's something important and potentially scary and something that will have a big impact on their lives. So the silver lining of that is that now we have many more people in institutions that are paying attention and helping in the effort to like make this go well. And so I think if it were just like a bunch of weirdos in San Francisco trying to make this go well, I'd be much more nervous. But because I think we have many more parts of society that are engaged now, including all of you, like that makes me more hopeful.

Let's see if we can squeeze in one or two more before, uh, we head down and, uh, have snacks.

Hi, I'm Sam. Um, I'm a master's student, uh, studying computer science and also some AI research. Also, before my question, I watch Hard Fork weekly.

So big fan.

Thank you.

Thank you.

But yeah, so you've talked a lot about like San Francisco kind of being like the center of gravity for a lot of what's going on in AI in the US and even just like the globe. And I think it's interesting now, like on the East Coast, maybe talking, especially like New York Times is based on the East Coast. Do you see like, what's the main kind of difference in the philosophy or like way of thinking about AI about people on the East Coast versus West Coast and even specifically like Academia versus industry? Do you like see, like, what's the main difference there?

That's a great question.

Um, and I don't think it's as, as clear-cut as East Coast versus West Coast. I think it's, I think it's probably more like San Francisco and DC are closer to each other in terms of like their attitudes toward AI. Like New York and LA tend to be like more aligned. So it's like sort of like a, Like a, yeah, there, that would be like more the axis I would draw. I think the, the, I think to crystallize it would be like, I think people in New York, this is obviously very broad brush, but like look at the floor of AI capabilities and people in San Francisco look at the ceiling. So I think people in New York sort of saying, oh, well, it got this thing wrong or hallucinated this fact. You can't trust it. And therefore I'm skeptical that all of this, the build-out, the billions of dollars being spent on AI infrastructure, like that is overhyped. This is just marketing. These tools aren't actually that useful. And I think in San Francisco and places like DC where people are looking at not only like the economic implications, but national security and governance implications, they're looking at like, okay, what could a really motivated person who's really skillful in using this tool do with it? Could they conduct an automated cyberattack? Could they build a billion-dollar company with one person? Could they do some transformative new— could they solve one of the Millennium Prize problems in mathematics, these sort of highest and hardest open math problems? So I think that's sort of the spiritual difference I see is like one group of people is sort of looking for the shortcomings and the gaps and one is looking for like When everything goes right, what can this thing do?

Last question, best question.

I appreciate you taking some time out here. I'm also a big fan of your Hard Fork podcast. I had a question about a recent episode that you did regarding talking about the AI and the, like, sort of the anger around it. In your podcast, you kind of chalked up a little bit about the anti-data center sentiment as being a sort of like NIMBYism. And you compared it with like the San Francisco housing crisis where like nowadays the prices of houses in San Francisco are like $2 million. But I also like to point out that like another case would be something like the interstate highway system where, you know, the roads were necessary and it was absolutely necessary for for the national place, but the communities that, there had to be a lot of communities that had to be demolished because of it and whatnot. So I guess I wanted to know a little bit about your stance about like where on this NIMBYism spectrum do you see the construction of data centers and like for those places that do have data centers be built in them, how can those local communities benefit for providing a national I just have to say, I love that Eric is taking my graduate class, which is on social media, but he's just managed to channel my honors seminar, which is about urban development and visions of the future.

So this is what you get from UMass students. They're going to hit all the rails on their shots.

Yeah, I got a lot of pushback for that episode, and I think it was fair. Look, I think Data centers are a stand-in for something that people are feeling that is real, which is that this technology is arriving very quickly. Their input has not been solicited on it. People don't feel like they have a lot of agency. Like there's people from San Francisco tend to talk about this as just like sort of this inexorable force that is just happening to people. And like, there's nothing you can do about it. You either like get on board the train or you get run over. And so I think people look at data centers as being like, this thing that is happening that is tangible and physical that is happening in their neighborhoods and that they actually can stop, like they can do something about this. I think I understand the frustration. I just think it's like not the right lever for change. Like it is not opposing data centers is not a stand-in for actual governance of AI technology. I think it is kicking the problem to another jurisdiction, to potentially another country. They will put the data centers somewhere. It is not like they are reliant on this being, you know, in one geographical area. So if they can't build them in the US, they'll put them in the Middle East, they'll put them in Canada, they might put them in space if Elon Musk has his way. And I think all of this is just, I don't think that it is actually a substitute for what we need, which is a government and a public sector that is paying attention to this technology that has a coherent strategy for regulating it. To me, it just seems like the wrong lever for change. But that said, it is the lever that is available to people. And so I don't blame them for grasping at it.

So Kevin, thank you so much for being with us. This has really been wonderful. Thank you everyone who's been here for some great questions. Thanks, Dean Keith, for making this possible, bringing us together. We're going to head downstairs. You may have a chance to talk one-on-one with Kevin. Let me give the mic back to Keith before we head out.

Let's have a round of applause.

Thank you.

So thank you so much, and please join us downstairs and maybe you'll get a chance to ask a question to Kevin one-on-one. Thank you again.

Thank you all.

This has been fun.