Google's James Manyika | Social Intelligence

Hi everyone. Welcome to Grey Matter, the podcast from Greylock where we share stories from company builders and business leaders. I'm Heather Mack, head of Editorial at Greylock. On today's episode, Greylock General Partner Reid Hoffman interviews James Manyika, who was recently appointed as Google's first ever SVP of Technology and society. James is a renowned expert on the impact of technology on various aspects of life. Throughout his career he's been particularly focused on artificial intelligence, robotics and globalization. He spent nearly three decades at McKinsey Global where he advised many of the top tech companies in the world. He's also held numerous board and advisory positions across academia, nonprofits and government. James spoke with Reed as part of Greylock's iconversation Speaker series. You can watch a video of the interview on our YouTube channel or read a transcript on our website greylock.com blog.

Hi everyone. Welcome to another edition of Greylock's @conversations. I'm Reid Hoffman, a General partner at Greylock and your host for today's event. I am thrilled to welcome James Minica as our guest today. James is the newly appointed SVP of Technology and Society at Google, a role he has taken on following more than 27 years at McKinsey Global where he advised leaders at many of the top tech companies in the world. James has focused on artificial intelligence, robotics and globalization from for his entire career and he's had extensive experience in pretty much every type of workplace imaginable, from academia to government agencies to private companies and nonprofits. His contributions range from books and articles to speeches, lectures, and of course, countless moments of critical advice in top secret capacities. He's held several government advisory posts including as Vice Chair of the Global Development Council under the Obama administration and he was named to the Digital Economy Board and National Innovation Board. He also serves on the Board of the Council of Foreign Relations and recently co chaired the State of California's Future of Work Commission. Today he serves in various science and technology capacities including as a Distinguished Fellow of Stanford's Human Centered AI Institute, Fellow and Visiting professor at Oxford, Board member at the Broad Institute of MIT and Harvard, among others. He is also a Fellow of the American Academy of Arts and Sciences and and a member of the National Academy's Science, Engineering and Medicines Committee on Responsible Computing. He's also a great friend of mine who have shared many incredible initiatives and experiences, including our recent trip to the South Pole where I believe James was the first Zimbabwean born visitor to the geographic South Pole. James, as always, thank you for being with us today. You've had a Long and busy career through one of the fastest changing periods in our history. And. And to this day, you are constantly expanding your exploration zone, not just to the South Pole.

Welcome. Thank you, Reed. I'm excited to do this with you. It's always fun to be in conversations with you.

So we were at Oxford together, but didn't know each other then. You've been focused on artificial intelligence and robotics for your entire career, including then from the technical aspects to the ethical aspects, which obviously super important for designing, implementing, founding the technology. What set you off on this course?

Thanks for asking the question. In fact, it's, you know, apart from growing up watching all the, you know, science fiction films, 2001, the Space Odyssey and everything, I had a very peculiar thing happen on as an undergraduate in Zimbabwe. So I was looking for an undergraduate research project and it turns out that a postdoc was visiting from Canada. This postdoc had been one of Geoff Hinton's students, actually. So he said, well, why don't you do a project building a neural network? And I said, what is that? So that was actually the first time I ever actually programmed a neural network. As some of your audience will know, Geoff Hinton was among one of the people who actually pioneered the successful run we've had now with deep learning and neural networks. So that was how I got started. So from that I got hooked, went to Oxford. At Oxford, I, you know, I did a few different things. But when I finished, I'd done a doctorate in AI and robotics. That was a fascinating time at the time, by the way. There was often a reluctance to call this AI because of the, you know, previous period of AI Winter. So we actually called it machine perception, machine learning and other things. Anything but AI.

Exactly. And one doesn't normally go from a PhD in machine learning, machine perception, to McKinsey. What was that, that move? And in particular, of course, you know, and we'll get into this in, in some depth, you know, kind of thinking about societal and economic ecosystems as part of this.

Well, it was a very accidental thing for me, Reed, because part of it was, quite frankly, an excuse to be in California, because when McKinsey made me an offer, I could be in California. And I'd been spending some time, by the way, Even after my PhD at Jet Propulsion Labs, when I was in the. A visiting scientist in the Man Machine Systems group, because some of the things I worked on, my doctorate were applicable then. And besides, a few friends of mine had this crazy idea that we might actually build an autonomous car. So my other Friend was at Berkeley and we're working Stewart Russell and others. I said, well, maybe, you know, this McKinsey thing is an excuse to be in California, be in Bay Area. So I actually took a leave of absence from the other stuff that I was doing just as an excuse to be in the Bay Area. And I guess I ended up staying. More seriously, though, I think part of what I learned at the time was that I was fascinated by very large scale problems. And of course technology is a big part of that. But also just think about large scale societal questions was what fascinated me and McKinsey seemed to be a great platform and place to do that, particularly at the Global Institute, where I ended up leading the Global Institute for many years.

Yep. Well, then the Global Institute is obviously, you know, producing a huge amount of very interesting and very practical, is not on the wrong word, but rooted in what kinds of things to do and what trends are happening. And so that makes sense. What's this new role at Google?

It almost feels like a continuation of the things that I've been passionate about, Reid, in the sense that it has this big title of technology and society. But what it's really about is to really think about what are all these advanced technologies going to mean for society in the best sense. Right. In the sense of what opportunities will be created, what things do you want to make sure we manage well. So I'm going to be spending a lot of time doing research, spending time with the amazing AI teams at Google, Jeff and Damis and others, and also, you know, thinking about the next generation of bets and investments and how those might affect society and quite frankly, talking to a lot of people inside Google, outside of Google, and trying to engage in these issues around technology and society. I, you know, you and I have spent a lot of time thinking about how do we make sure that things go right. And I'm particularly excited about making sure in an area like AI and robotics where I've spent most of my time, I want to make sure it turns out right.

Yep. So one of the things that you have coming up is through the American Academy of Arts and Sciences, where both you and I are members, they have a magazine AI issue coming up to kind of begin to dig into these issues and some of the things that obviously you've been doing at McKinsey and we'll be doing at Google, what are some of the issues that you're going to be addressing in that issue of Daedalus and what are some of the things that you think? Because I think we're probably broadly more talking to technologists here that technologists should be thinking about as they think about this amazing renaissance we're going in. AI, which we will obviously dig into.

Well, first of all, I was so excited when the American Academy asked me about a year ago to curate and edit this special edition of Daedalus. Normally, when Daedalus comes out, it is like eight essays, but this one's going to have 27 essays. What's fun about that is I was able to arm twist my friends and people I know. So I've got everybody contributing an essay from Jeff Dean as an essay. Kevin Scott at Microsoft has an essay. Mira Muradi from OpenAI as an essay. Stuart Russell from Berkeley. So you got to half the contributors are sort of AI pioneers and people who really work in the frontier of the technology. The other half are people who are thinking about the implications on society. So they have got people like Eric Brynjolsson, an economist, Michael Spence, a Nobel laureate, philosophers and ethicists. So this issue is actually called AI in Society. But if you come to your question, Reid, about where are we in this? I think on the technical side, it's a very exciting time. Your audience will know and the participants will know that we've had an incredible run with techniques like deep learning and reinforcement learning, and especially in deep learning, now you got a turbocharged recently with the development of transformer models. So these systems are working remarkably well. One of the debates you'll hear in the collection, and it's very much in the community, is are the current approaches sufficient to get us to remarkably powerful AI or even ultimately artificial general intelligence? So this is one of the debates in the field which is, are these techniques and approaches enough? And if you take a room of AI people, half of them will say, oh, yeah, this is all we need. And if you talk to the other half will say, well, this stuff is great, but we need so much more. Usually what they're getting at there is there's still some very hard problems in AI. They range from things like can we actually do causal reasoning with these systems? Can we get to issues of meaning? Can we do transfer learning? Can we do what, you know, Daniel Kahneman and others described as System 2 tasks. So there's still some very hard problems. And the debate is, are these techniques enough, or are we going to need other things? By the way, that debate is typically when you ask people about AGI, about how far away is it? Much of that debate hinges on this question. So the people who think we've got all the tools and techniques will say, oh no, it's very, very close. The people who don't think we've got enough will say, no, no, no, no, we still need some major conceptual additional breakthroughs before we can get there. This is one of the fun debates and you'll see this in the collection, at least on the technical question. Now of course, there are other questions that are being debated in the edition I'm sure we'll get into about some of the societal issues, the economic jobs implications. The fun topic you and I have fun with all the time, great power competition. Right? How do you think about how this plays out on the global stage? But it's going to be a pretty broad, fascinating discussion.

Yeah, let's use that as initial discourse. Because one of the things I think sometimes gets lost in this discourse is you say, well look, there's two camps. One thinks that AGI is, you know, 10, 20, 30 years in the future. And there's always this little betting pool of where do you get to 20% or 50% or 80% as a chance and number of years? And you and I both run that little opening poll question in groups rooms. But even if you don't get to AGI, the transformative impact of, you know, this new renaissance in AI is going to be huge. Say a little bit about kind of what you see as coming even if you don't get to AGI. We will get to another thing on AGI and what are some of the kind of societal questions that technology should be thinking about and part of we as a broader discourse. So what's possible even with just machine learning as it is? And then what are some of the important questions for us to address?

First of all, there's just the economics of it. So one of the things that is actually truly exciting is the amazing transformational potential of these technologies to economic growth and prosperity. A quick way to see that is 30 seconds of kind of economic theory. So if you look at how the economy grows, people got Nobel prizes for this. There's something called growth disaggregation that Bob Solo and others got. So you get GDP growth by either getting productivity growth and labor supply growth, you put those together, GDP growth. Now with aging and other things, much of our forward looking economic growth is going to come from productivity growth. And, and at the core of productivity growth is technological innovation. And at this point in time, AI and related technologies have extraordinary potential to transform how productivity growth happens. So in that macro sense, we need it for the economy and economic growth. You come Back to a much more practical level, we did some research with friends at Google and Microsoft and a few others to look at use cases in the economy. So coming down from the lofty kind of economic theory level, you come down to actual use cases. And if you look at these techniques in the, you know, we looked at the initial where we looked at something like 400 use cases. Now that library has gone up. Now I have thousands of use cases. What you find is that the powerful, economically compelling and commercial use cases are actually there in every sector of the economy. So this is not just something in the tech sector, but it's everywhere. And the biggies, by the way, are in retail and transportation logistics, just from the sheer view of applicable use cases. And it's not just a sectoral question, it's also a functional question. Sales and marketing is actually one of the biggest arenas, supply chain and logistics, if you think about it as a function inside companies. So the use cases are there everywhere. In fact, when we first tried to size this read, just looking at the available, what looked like the realistic use cases, we got to a number that was at least $6 trillion potentially annually in the near term. And if you scale that up to all the other use cases that are emerging, the numbers get bigger and bigger. So in a nutshell, the economic possibilities and potential, both for the economy, for businesses and companies, are tremendous. Now, the other question that comes up, of course that is an implication even before we get to AGI, tends to be the question about jobs and labor markets. This is where the story is a bit mixed in the following sense. So while the economic case is clear, the jobs, labor markets question is a bit mixed. It's mixed in the sense that we've done lots of studies as of others too. And generally what most of those studies conclude is that, first of all, don't worry about a jobless future, at least not for the next several decades, because what will happen is we're going to lose some jobs, but we're also going to gain some jobs. But the biggest effect is that most jobs will change. But what will happen, therefore, is even though we have jobs for the future, the question is how do we deal with the transitions and the transition issues here have to do with reskilling, with occupational changes, with some of the wage effects. And these are the things that we're really going to need to navigate and think through quite carefully. But for the most part, don't worry about a jobless future, at least not for the next few decades. So that's the economic, the labor market question tends to be a little bit more complex, so we can discuss that some more. Now, there are other effects people think about. These are the societal, ethical and other questions, and I'm sure we'll get into those because these transformational technologies have a way how we use them. Questions of use and misuse, questions of the second order effect. When it comes to things like information disinformation, deep fakes, etc. There's a whole other set of implications that we have to think about. It gets to questions of governance, use and misuse. Then of course, we get to your. One of your favorite topics, Reid, which is just how does this play out in terms of great power competition? We know that already two countries are racing ahead, ahead of everybody. That's the United States and China. Others are not quite, you know, progressing at the same pace. So how does, what does this all mean when you've got so much economics at stake, but also these national security and geopolitical strategic interests at stake? That's another whole question and arena for this field.

Yep, I totally agree. Obviously, given the number of conversations we've had. I mean, I think part of the frame of this is for folks to realize that even though there's been a lot of drum rolling and as yet relatively modest industrial impact from the AI stuff, as the software transitions both into the kind of, one of the frequent expressions I use, the world of bits and atoms, we're going to see some particularly massive changes. And, you know, it'll probably be a little bit more surprising and a little sooner than we think. You know, some things will obviously, you know, like, for example, jobs I think, will stay. The jobs will transform, but there will be jobs. It's just the transformation, as you say. One of the things that is obviously, I think part of what makes everyone nervous about the AI technology is the explainability is the transparency kind of problem, which is the what is this device doing? So maybe this device will drive the vehicle, but like, will it make sure that it will drive it safely and do you know it and how do you get a sense of it, given how complicated, you know, hundreds of billions of parameters in these transformer models? Do you see, kind of how do you, how can you bring other people into the discourse and how do you get other people to get some confidence about what's happening, how to put those two elements together?

That's an important question, Reid, because I think, you know, despite, you know, our collective exuberance about these technologies, there are some major limitations and as I said, some hard problems But I think on the, it's worth dwelling a little bit on the limitations and gaps before we get to the truly hard AI problems. I think the limitations include things that are likely to erode public trust. A lot of those include things like bias in algorithms or in the data or the corpora that I use to train them. Questions about brittleness, for example, in the systems, this issue of when you, when you have out of band distributional non stationarities, for example, when you, you know, when you train the data sets on this, on these set of things and then you suddenly present them something out of distribution, it makes different predictions. And also the issue of explainability because from a trust building standpoint. Now often what I often find interesting, Reid, is that often people who don't understand the technology outside the tech industry think that the tech industry is trying to hide something. On explainability? No, it's just that the neural network structure, the structure of these algorithms are such that you can't actually open it up and say it made this decision because of this particular variable, that particular variable, or this data set or although we're starting to get better at that. So the question is how do we address these limitations and gaps that are likely to erode public trust? And I think it's important to keep public trust in these systems because if these systems are going to show up in health applications, in autonomous vehicles, et cetera, people need to understand and be trustful of these systems. Now how we get to that, I think there's a lot more research and work to be done and I think some of that is underway. I think we're starting to make progress on that front. But also I think having the public keep up with the field and understand how these systems works, there's a lot of education, a lot of involvement and participation in the processes that are going to have to. But this is one of the things we have to get right. We have to continue to build public trust in these systems and be quite open and transparent about what we know and what we don't know. So those are some of these limitations we need to work through technology despite, in addition to all the hard other problems we still have to work on. There's a lot of work to do. And by the way, one of the, I should have mentioned one of the other trust eroding limitations are some of the toxic outputs from sometimes for these algorithms which happen not all of the time, but some of the time. So this is one of the criticisms, for example of large language models. I'm sure we'll get to talk about large language models, but that's one of the criticism that occasionally you'll get sexist or racist outputs from these systems. We need to fix that. But that, that's, you know, that's one of the, these trust eroding issues with these systems.

Let's go to the large language models, which is one of the areas that generating. And every, every group that I know that's working on them is now very focused on questions like racial bias and you know, alignment and safety questions in part because they kind of reflect the data that's there. And it is to some degree a partial Rorschach test on our society, but also of course also on where the data is. It's a little bit like kind of classically in the criminal justice system. One of the reasons why an algorithm that goes across the data set that's reported captures the fact that, you know, over the last decades police forces have, have more policed the communities of color than they have others. And so they have all the data that then gets enshrined. And it's one of the things you have to be really careful about in things like risk and, and parole and credit and other kinds of things that are, that are super important. And I think one of the really good things is everyone now goes, yep, we're doing something on it.

In fact, I think one of the things about these issues of bias which has showed up in yes, as you said in criminal justice system, in hiring, for example, often the issue is with data and often the issue is how that data is collected. And often the issue is actually not with the algorithms, quite frankly, but rather how society has set itself up to gather this data. And policing is a good example because we know, for example, there's greater police presence in certain neighborhoods and places, so more data is collected. So are you surprised therefore, when predictions are made from areas where there's they heavily whether a lot of data's being collected that will make certain predictions, of course not. Similarly, you see it on the other end of the spectrum in financial lending where if you've got people where it's the opposite problem where if you've got people who are quote unquote off the financial grid, often algorithm make bad predictions about whether they're worth lending to or not. Whereas people for whom we know a lot about, we tend to make better predictions about. So these questions about how society itself captures the data is as much an issue as it is for the algorithms. Now that's not to say that we shouldn't think about ways to spot that in the algorithms and how we curate the data. We should. But quite often these systems are actually highlighting our issues, quite frankly. I mean, one of the fun things, as you know, on this fairness and bias, you may. I think you saw this paper, Reid, where some AI researchers tried to say, okay, so please society, give us a definitions of fairness so we can actually build algorithms that do that. And I think in that paper they identified 21 different definitions of fairness. So how do you even create algorithms for that? I think so these systems are actually forcing us as society to really ask ourselves some really hard questions. I don't think we've ever fully defined fairness. I was actually at an interesting symposium which was included AI researchers, sociologists and lawyers and philosophers. And the general conclusion, by the way, in that discussion was of the form. You know, society, we've never really defined fairness. We tend to have to use two proxies for it. One proxy that we use is we say, well, you know, what you might call procedural fairness. So in other words, if something's gone through this process, will assume it's fair. Right. Just because it's gone through this process, we'll take the output as fair. Or we'll say what's sometimes referred as compositional fairness. We'll say, okay, if this group of people made the hiring decisions, we'll accept what they say because the group is made up of these people, this particular composition. But you notice in both cases, we've not really defined what the actual fair outcome looks like. We've defined a process that we take as fair. We've defined or just trying to composition to make the decision that we think is fair. So in other words, we're still ducking the question what does fair look like?

And obviously part of the challenge and fairness, which is the reason why that excellent paper, which I think I first read because you sent it to me, it's somewhat political and human groups conflict and they say, well, I want this definition of fairness because it's better for me. I want this definition of fairness because it's better for me. And it's part of the reason why these things are always complicated. What is some of the best ideas that you're seeing for how to deal with, call it broadly, these corrections of societal biases? Because I think one of the things you and I share, although we've been in groups that don't have this point of view, is that, look, one of the benefits of putting this in an AI system is we can fix it. We can make the system better. So you go, well, okay, it's current software, algorithmic systems are doing better bad things when they are making parole recommendations or when they're making financial credit decisions, they're doing so in part because of just the data that's there. What are some of the things that you're seeing that are like, okay, this could be a good fix because then we could make society better because we fixed this underlying bias that might have been there anyway. And so, as opposed to being institutionalized by the new AI software system, it can be improved. What are some of the ideas that you're saying that are the best things to implement on that?

First of all, just to note, you know, acknowledge the reason, there's a good reason why people are concerned about algorithms in this sense, because even though they have the potential to have to do a lot of good, they tend to be what some have called formalizers and amplifiers. Because if you get the algorithm wrong, you're going to deploy it at scale and you're going to bake it in. And so the effects in some ways may be much worse than a single biased judge, when in fact the whole system of judges are now relying on the system. So this question of formalization and amplifications is a legitimate concern. But to your question about what are some of the ways that are getting at these things, I think there's some very cool technical work that's going on where people are building ways to actually examine data sets and actually spot structural biases in the data sets themselves. There's some very, some terrific work that some researchers, Sylvia Chiappa being one of them, that are doing on things like counterfactual fairness, which is, how do you actually build, you know, if you like competing algorithms that actually try to understand and do counterfactuals on the data, to say, if we took this part out, could we have made a different prediction if we take this part out? So you can imagine having AIs actually help with that problem is actually one of the clever ways to actually spot the biases in the system. So I'm actually excited about that kind of work that's going on, but I think the other part is just actually having different kinds of people be involved in the process. So one of the things that I've been involved in, I think you mentioned at the beginning, there's the National Academies of Science, Engineering, Medicine, put together this committee that I've been on on responsible computing. And you suddenly realize that one of the things that happens is you, you need people to Ask the right questions, having different multidisciplinary people, diversity, especially in a disciplinary sense, who can actually ask the counterfactuals in the development process that can actually make a big difference. I should point out though, by the way, Reid, that one of the other fun debates, by the way, in the field is much like there's a fun debate about are these techniques sufficient or not. The other debate has to do with approaches to developing these systems in the following sense. There's one camp that feels we can correct and improve these systems if we put them out into the world and have people use them and capture the errors, etc. So if we do that at scale, having real people use them, that's the way to improve them. The other camp says no, no, no, we need to test and test and test and test and test and test before we deploy anything. So you're seeing this play out in the real world, even in approaches to autonomous vehicles, for example, where one school of thought is put mountain to real systems and learn, others say no, no, keep doing training runs and training runs. So this is one of the questions and competing approaches to this question of how do we get these systems right.

Yep, I completely agree. One of the things you gestured at earlier, and this has obviously been in the dialogue for a number of years, obviously played to your California's future work commission was the Silicon Valley folks tend to go oh my God, the future's.

Going to be here right away.

So like for example, we're going to have this isn't yet getting the AGI. This is the transformation of work. They tend to say oh my God, you know, it's like a Star Trek future. Robots are going to be generating everything we need, universal basic income, jobs are going away, et cetera, et cetera. Say a little bit about the actual work you've done looking at this, what the actual shape of the next few decades look like so that we can predict and intervene in ways that are good for society, whether it's building the technology understanding policy. But like what does that future of work look like?

This is where we've got some really hard questions. I mean the commission you're pointing to, this is an interesting arrangement. So I was co chairing the commission with my co chair was she's an extraordinary leader of one of the largest labor unions. And on the commission itself we had business people from the labor, people from academia. So it's a very, you know, technologists and others, I mean Fei, Fei Li was on it. So it was a very diverse group and when you look at these questions about the future of work, I think I would pass the issues in a few categories. One is the issue around just the fact the reskilling. The reskilling one's an important one because we know that jobs are changing. Many more jobs will change than will be lost, actually. So having people be able to adapt and keep up with those changes is a really large problem. And in fact, one of the issues about reskilling is, you know, one of the fun quick exercises you can do, Reid, is to anybody tells you, oh, we're doing reskilling, right? The next question you want to ask is, okay, so how many people? And what you typically find that it's been very hard to do reskilling at scale. So people may say we reschooled 100 people, maybe 1,000 people, but the scale of the reskilling that's going to be required over the next decade or two is actually in the millions. So you've got those kinds of issues. The other issues that we have are the wage effects. And let me go into a little bit of detail on this. And we've seen this play out in the real world. So the good thing is that I think technologists are correct when we say these AI technologies are going to complement as opposed to replace people. That's generally true. But that often can have both positive and adverse wage effects in the following sense. So, for example, imagine a technology that complements you, Reid, or complements me, or complements a radiologist. That actually is great. It makes you and I and the radiologists way more productive. And everybody benefits. We are more productive, we earn more money. The outcomes are good for everybody. People get the benefit of outputs. So it works all out. It works out, that's great. But look at the other end. When you complement, for example, some more basic occupations, in some cases, the technology is actually complementing and doing the value added portion of that work. And what's left over for the human may be the less differentiated aspect of that work. And what happens as an economic question is that what you've just done is expanded the labor pool available for doing that. So in other words, the classic example is the London black cab example, where decades ago in London, to be a black cab driver, you had to literally memorize the map of London. And there's a test called the knowledge. The knowledge, exactly. You had to pass that. And there have been some very good studies that have actually shown that with the advent of GPS systems, for example, it literally took away and had an impact on the Wages, because all you now needed to do to be a really good driver in London, just know how to drive. Because gps would solve the knowledge part of it. And so what that does, it expands the labor pool available and in fact can have a depressive effect on wages. So we have to think about these wage effects. And that's typically, which then takes you to the, sometimes the UBI conversation, because what happens is it's a recognition that on the one hand, we're creating, as I suggested at the beginning, enormous economic potential. So if we're going to create all this abundance economically and yet we may have these adverse wage effects or worse, shouldn't we be somehow creating a way for people to generate income? That's where the wage, the UBI question typically comes from. Now, personally, I'm not a proponent of ubi, but I like the UBI discussion in the following sense because it's, it's getting at a real question, which is what happens when we're creating economic surpluses and yet wages are not going up as much for everybody? What do we do about that? So I like the question for the debate it provokes about the wage effects, which, by the way, are already here. So one of the things we've learned from the California Future Work Commission and other studies across the country is that we already know there's inequality. We know that technology is one of multiple factors contributing to that. So the immediate question we're going to need to deal with is not are we going to have any jobs? But are the jobs going to pay enough? And that's a very real question you see in all these commissions across the country and other countries, especially in the advanced economies. It's a real question.

Yep. And clearly we're already facing some of the things in that. Although I think personally, and I'd be curious your point of view. Most of the things that when people are saying are currently wage effects of automation or more wage effects of globalization, the automation is still very much coming and is important to focus on. But it's like, well, the robots are taking your jobs. Like, well, I'm not sure. Actually the question is yet if the robots are taking your jobs as much as can the robots get here soon enough to empower higher wager jobs and can we get the higher wage jobs part of it is the, the more relevant part.

I think that's right. I think the globalization in jobs and wage question was a legitimate and real question if you looked at the late 90s, early 2000s, because that was in fact the driver of some of the wage effects we saw at that time. But I think that trend is going away because we know that pretty much everywhere, including in places like China and others which were playing the labor arbitrage game, the wages are going up there too. And so the dynamic going forward is less on wages, is less about globalization. By the way, if you looked at, if you go through and typically the globalization wage question plays out often in manufacturing and other service sector jobs and what happens there. If you looked at now, the only places where you still have some extraordinary labor arbitrage going on tends to be in a few narrow parts of manufacturing, typically furniture manufacturing, some portions of textiles. It's not everywhere. Large parts of the manufacturing sector which include automotive, chemicals, etc. That effect is not really at play anymore. So the on a forward looking basis, I don't think the debate on wages is about globalization. It's about the structure of the economy. We know that when you have an economy that is service sector dominated, which most advanced economies are ours, certainly is the way that you question is very real. I mean we can talk about manufacturing all we want. Today manufacturing is 9% of the US labor force. It's not the dominant piece. The rest of it is everything else, including the service sector economy. The last time manufacturing was a big part of the economy was 1958 was the peak of it. It's been coming down ever since.

Interesting. Yeah. Although obviously the classic problem is the service economy absorbs a lot of people in terms of number of jobs, but don't really get the leverage to get the wages up on that is actually in fact an interesting challenge.

That is correct. And by the way, you know, I'm kind of being somewhat flippant about manufacturing its size and scale. One of the nice things about having a manufacturing sector, it has these multiplier effects around it. So we know that when there's manufacturing activity going on, the multiplier effect on adjacent services is actually a positive one. So there is a need to actually have manufacturing. Not that everybody's going to work in manufacturing, because I don't think that's going to happen. But it has these wonderful multiplier effects. But you're right, that's the question, which is how do we think about wages and income in a service sector driven economy? In our portions of the service sector are the lucky ones. The rest looks like people work in restaurants, people who work in services, etc. The wage structures that are quite different.

Let's shift to artificial general intelligence now. So as you're doing in DAEDALUS and, and as you know, we have a wide variety of AI researchers, some who are like, look, we've got fundamental things, the things that you gestured at before, you know, everything from symbolic reasoning to, you know, kind of language, one shot, learning, transfer learning, etc. As like, no, these things have not been, you know, we have fundamental innovations coming. We have other folks saying, no. In fact, the scale of these language models or the foundational models together with some other innovations will actually, in fact make this stuff work. And so, you know, these are the things that are kind of playing into this. And so what are some of the observations you might share with our community today, our group on kind of how to think about AGI soon, Possibilities, probabilities, constraints?

First of all, just start with large language models. I mean, these have been remarkable. I mean, I think with the, starting with the advent of that classic transformer paper that Vaswani and others at Google did about three and a half years ago, that has led to these, you know, you know, everybody's building a large language model now. I mean, Google's built, you know, went from BERT to now there's Lambda, OpenAI has went from GPT 1, 2 and 3, and there's more to come. And Microsoft is building MTN, LG and DeepMind is built GOFA. So there's just bigger and bigger models being built. The performance not entirely seems to improve with size, but there may be some limits to that. And what's remarkable about large language models, as you know, Reid, is the fact that they've also been able to have these multimodal outputs. So you can go from natural language. Yes, to natural language outputs, but also do natural language to software code. Microsoft and OpenAI did this, you know, went from GPT3 to codecs which generated software code. You can do natural language to images. So that's what's often led to people to start to think about them as foundational models. So the question is, are those a way to get to things that start to look like general intelligence? So I think people working in deep learning, people work in reinforcement learning. By the way, reinforcement learning is what you've seen DeepMind and others and OpenAI do quite a bit. DeepMind is taking it to do all the alphas, right? Alpha, zero, alpha, fold in doing strategy, science, et cetera, et cetera. So I think what people generally conclude is that these approaches still have a lot of headroom, there's still a lot more. We have not reached the limits of these techniques yet. And these approaches and we're Going to see many more innovations. But you come back to this question of is that enough? I happen to think that that may not be enough actually because, you know, there's still some really hard problems about causal reasoning and meaning and understanding about, you know, are we actually building in understanding and generating understanding. We still have a hard time doing science, you know, we still have a hard time for example, generating systems that create kind of novelty or conjectures, say in mathematics, for example. Although there's a fun recent paper on knots, on knot theory and topology that came out of DeepMind, which is kind of fascinating. But these are still very hard questions, questions about memory and persistence. We don't quite know how to do that. In fact, many still think that there's a lot more we can learn from neuroscience that we haven't fully mind what we can learn from neuroscience about cognitive agents and models and how that works. So to declare my own biases, I tend to, I think in the camp of we need more, we need more conceptual breakthroughs, I think before we get to AGI, but at the same time I think we're going to get a lot more out of these systems that we're building today. The question is at what point should we start to prepare for the possibility of AGI? And I don't think we need to go all the way to AGI before we start to have systems where we need to be thinking about the implications. I think it's one of the key questions, quite frankly for the field, which is how do we prepare for that, how do we coordinate. I think one of the amazing things that has happened is that the players and the researchers at the forefront of all these developments are the good folks. So far, I think with the best intentions. Of course they compete intensely, but I think they're all trying to want the best for society. I think they've written this into their missions, et cetera. That's good, but I think that may not always be the case. I mean these systems can become so powerful. And so we need to start to think about how do we approach questions about safety, how do we think about questions of control, questions of human alignment. Stuart Russell has been, you know, press at Berkeley, you've read his book and others may, may have read it or find it interesting. On human compatible AI, this is the question of how do we build provably beneficial systems that are roughly aligned with, with our interests and what's best for us. And this is a very hard, both technical, scientific question, but quite frankly societal question. So I think these are some of the hard questions. But, you know, as I said, I'm of the view, Reid, that I think we need more.

Yep. I think even the most sophisticated people on deep learning think that there needs to be some more, too. The real question, is it this much more or this much more? Like, how many innovations is it in order to get there? One of the things that you gestured at, very importantly at the very beginning of our discussion day is that it is going to be essential to build and maintain public trust, because these systems are going to have a huge impact on people's lives. The economic system, jobs, wages. There's a variety of places where they can intervene. What are some of the things, do you think? We as technologists and we as tech companies? Because one of the weirdnesses in the public trust is if you look at the current activity, it's all massively being driven by corporations, actually, frankly, both within the US and within China and other places, all corporations. And obviously there's some worries about corporations doing this. There's profit motives, there's some societal questions that are more urgent to society and individuals than they are to corporations. Fairness, for example, might be one of the things that people would worry about. Or what are some of the ways that we as technologists not only should be thinking about it, but engaging in public dialogue and doing things that would help build and maintain or even rebuild and maintain. Because I don't think public trust is at a current high watermark in order to be doing that.

This is a very hard question, Reid, and I think it's actually one of the things that's complicated about AI compared to other technology, which is it is being led primarily by the private sector, as you said. I think a few things come to mind. I think one is the sector itself, I think, has to be more transparent about how it's doing what it's doing. And I think actually the public would actually be quite pleased. And one of the things that I always find interesting, and this is even before my role at Google, this has always struck me that when you spend time with the people actually building these systems in the current leaders who are developing these systems, pick any one of them, they're actually very thoughtful about what they're doing and they can debate about is this approach better, but you'll find that they actually care about the ethical implications. They may just be debating how to achieve them, but much of what they're doing is not as available and transparent to the public. People don't know that these discussions are happening, these debates are happening, these issues are happening. So they assume that it's all entirely driven by the profit motive and that's not true. So I think there's just something about more transparency which actually would show that a lot of thought is going into these systems. That's one. The other is, I think some involvement by the tech sector and the developers with policymakers and regulators. I mean, I'm always surprised by how little understanding there is in regulatory policy making circles about how these systems work. You don't just have to listen to hearings you see on C SPAN and others to see, oh my goodness, they really don't understand this stuff. So I think there's just something about the continual education, involvement and engagement with policymakers so that all of us and people are going to be making decisions about these systems. Understand that. I do also think that there's some place, quite frankly, for some peer pressure and peer group pressure in systems. One of the things I was quite pleased about a couple a few years ago was when the Partnership on AI was created, which is intended to be a peer group of leading companies who are leading research in AI, coming together, trying to establish their own set of standards. I think here there's a lot to be learned, by the way, from what happened in genomics and biology. There's a famous Osilomar symposium in the 70s and 80s that actually created a peer set of rules and principles about how to do genomics research that have largely held. And I know you've been a part of this read. I mean, this ecosystem is attempted to do those things. I think we need more of those kinds of things. But even that won't be enough. I think in the end we may need some clearer rules of the road. We may need to do that in a global sense because it's not just about the US and about Silicon Valley. These systems are being built everywhere. I think that's what was a little bit complicating because you've got a few different competing things going on at the global stage. One of the purely economic stakes and the economic stakes, by the way, are not just for companies, but they're for countries too, right? So the economic stakes affect countries and companies. Then you've got the national security stakes. I mean, you and I were on this task force on innovation and national security and you could see the tensions on that task force, right? You've got innovators and economists on the one hand saying, oh, but this is good for society and the economy, right? And then people coming out of a national security background by saying, but no, no, no, this might not be as good for our national interest. So the question is, how do we navigate those tensions? And I think one of the key differences between AI and other technologies that have had national security implications, like, I don't know, nuclear science, for example, is that the nice thing about nuclear science is that the governments typically lead in that. And so there's an alignment that it's governments who've largely led in that work in the, in the nuclear age, in the Cold War, and all the capabilities are on the government side. So you had an aligned center of interest. And the nice thing about nuclear technology, by the way, is that if somebody uses it, we can detect it. AI is a little bit different in the sense that on the one hand, all the innovations are in the private sector. The private sectors think about global economic opportunities, not just national questions. And so you've got that dissonance. The other dissonance you've got is the detectability accountability question is much harder with AI systems in a way that it isn't with say, biological weapons or nuclear weapons. So this is a much more complicated arena and there's a lot to think about.

Yep. We have a question from one of my favorite people, Selena Tabakiwala, which is, I think a classic and important question here and applies across a wide variety of fields. This one in particular, you know, you say, well, sexism, the bias and the algorithms data, how much of it is just better measurement and validation and how much of it is the people doing the work who will have a natural validation, cross check understanding about whether or not they're on track or not. What do you think that these components need to be in order to improve our alignment and the outcome of these algorithms? What's the balance and play between these?

I think you need both, and I would actually add a couple more when I say both. I think making sure we diversify the people involved in the development and questioning of these systems. And here diversity is important not just in the gender or racial ethnicity sense, but also just in the disciplinary sense. Because you find that sometimes social scientists ask different kinds of questions. Actually ethicists will ask different kinds of questions. So I think the question is correct. I think there's something about the diversifying the people involved in the development of these systems. I also think that there's also the other part, which is better thinking, curation of how we train these algorithms that comes down to the data question, quite frankly, and how society collects, aggregates the data. But even there I'd also add, you know, the technology itself can Help too. Right. So on the data front, I'd say we need to solve for society how society collects this data, as in the policing example, for example. But we also need to think about can AI itself actually help us spot those things, spot those patterns. So in other words, AI can actually be a tool to spot bias. And actually, in fact, you're starting to see examples of systems that try to do that, particularly with toxic outputs. For example, one of the nice things that some of the latest large language models do is that they're starting to do their own pre checking of their outputs, using almost adversarial, different other large language models to do that. So the AI itself can be a tool for that. I think we're going to need all these components. Now, how much you put weight on one of those elements versus the other? I think it's very arena dependent in the work that we're doing for the National Academies. How you think about that question in commercial environments where you got market forces at work is very different than how you might think about it, for example, in health and safety sensitive arenas like in healthcare or transportation systems, for example. So I think there's something about tailoring the mix of those things to the different settings and arenas. But you need all of them, I think.

Yep. Well, we have time for one last quick, fun question since we've gone through these very, very serious issues. I always recommend everything you write and everything you do because it's balanced and thoughtful and brings in considerations and is erudite and has depth of accommodation of scholarship, but also understanding, pragmatism and usefulness. What things would you recommend to this audience about, like what stood out for you lately? Books, podcasts, movies? What would you recommend? No, no, I wasn't looking for that one.

No, no, no. But it's true, right? It's true. Gosh. I mean, I think in terms of, you know, around the topics that we're talking about, I actually think your audience might find this collection of Daedalus interesting when it comes up because. Partly because it's an amazing group of people writing it and have very, very different views, by the way. And you'll see the debates when you read it. That's one. I also think that there's a. There's a wonderful book that Eric Schmidt and Henry Kissinger put together and Dan Hans Artenlocker which tries to get at some of these geopolitical questions that's actually worth reading and it gets a lot of the issues. And quite frankly, there's some really good papers. I mean, One of the nice things about AI field is that even the technical researchers have actually started to write good, approachable, readable papers. So I often think for anybody listening interested in deep learning, the paper that was written last year by Joshua Bengio, Jeff hinton, and Jan LeCun, who are kind of seen as, you know, the three of them won the Turing Award for their work in deep learning, they were kind of taking stock at deep learning where they feel where that particular set of approaches stands and what more needs to be done. It's actually a very good and very readable paper. So there's a bunch of things like that, Reid, that I think are interesting 100%.

So that brings us to the end of our discussion. James, thank you for being with us today. As always, I would take this show on the road any day, as you know.

Well, thank you. You and I still have a lot to debate and argue about with all these things, so we will be doing.

More in a variety of circumstances. And I think everyone has seen we've had a lot of people logged in a lot of our treasured members of our Greylock community here because they are looking for expertise and insight such as the one you provide. And thank you again, James, as always, erudite, insightful, comprehensive and deep. Thank you very much.

Thanks for having me, Reid.

That concludes this episode of Gray Matter. If you liked what you hear and want to find more interviews on entrepreneurship, Please subscribe at SoundCloud, Spotify, YouTube, or wherever you get your podcasts. You can also find all Gray Matter content at our website, greylock.com and you can follow us on Twitter reylockvc. I'm Heather Mack and thanks for listening.