Reid Hoffman and Saam Motamedi | Introducing the Intelligent Future

Foreign hello, and welcome to Gray Matter, the podcast from Greylock. I'm Chris Yeh and I'm here with Greylock general partners Reid Hoffman and Sam Motamity. Today we're going to talk about artificial intelligence. Now, Greylock has partnered with quite a number of startups in this rapidly advancing field, including Abnormal Security, Inflection, Adept, Snorkel, Neuro, and Cresta. Reid and SAHM have led many of these investments and have a unique vantage point on developments in AI. This discussion is a precursor to a series of conversations Reid and SAHM will be having with top experts, visionaries and early stage innovators across the AI ecosystem. They'll discuss everything from the latest achievements in robotics and applied machine learning to large language models and the ethical considerations of AI. And of course, they'll explore the many new opportunities for startups within this field. Guests of this series will include Microsoft CTO Kevin Scott, Stanford professor Dr. Fei Fei Li OpenAI CEO Sam Altman, and the founders of some of the most promising AI startups today. All of these conversations will be available in the coming weeks on the Gray Matter podcast. So gentlemen, let's start with some of the basics. What are some of the biggest technological developments in AI in recent years and what do they mean for the state of the field?

Well, one of the things that's pretty interesting here, Chris, is that there have actually been a number of really important ones, like the whole artificial intelligence field is a renaissance. My undergraduate major was symbolic systems at Stanford, which was kind of symbolic intelligence and cognitive science. And while some of those same techniques are used in deep learning and neural networks and that kind of thing, there's also been a huge number of new things. And so even though I'm going to throw out a few, it's just beginning. So one of the really central things was creating a new set of paradigms for training these models. Generally speaking, you know, multi billion parameter can be 20, can be 250, going on 500, et cetera, you know, kind of large network models. One part of the training paradigm is this thing called Transformers, which is roughly take a lot of data and text and then using the removal of interesting words, start predicting that that becomes a generative model. And that kind of generativity then creates all kinds of interesting possibilities. Not just the generational language, but a bunch of other things. But by the way, part of the reason I start with training paradigms is, you know, kind of goes all the way back to some of the DeepMind stuff that demis Sabas and Mustafa Suleyman did, which is like game on game, like training it by, like playing itself in AlphaGo or chess and other kinds of. Or Pong and ways of doing things. So there's a whole bunch of supervised and partially unsupervised and unsupervised training paradigms. This leads to large language models and foundational models, which is where a lot of the amazing magic currently tends to come from. And then there's image generators which use diffusion and other kinds of models, most notably with OpenAI's Dall E that create pictures that even when you say Future City in watercolor, it will actually give you a watercolor image of a Jetsons, kind of like future and so forth. And I recently did some essays on this and, you know, kind of release some NFTs. Because another thing we do at Great Lock here is also look at the web3 stuff. And so it's kind of putting these together as part of it. And those are just highlights. But that's just the beginning.

Yeah, just to tack on, I mean, Reid called out a couple of the really important trends here. One of the things that's been interesting to me is just the focus and innovation we've seen around Generative. Like, I think if you, if you rewind several years ago, a lot of the focus was on discriminative models. So show me a photo and tell me, is this a cat or a dog? And, and that's interesting. But like we talked about Dolly on the image generation side, we've seen models like GPT3 on the text generation side. And what's interesting is we're now beginning to see the applications of Generative actually go mainstream. Right. So today, if you're a copywriter, you can have AI help you actually dramatically increase the speed at which you develop copy, whether that's marketing, copy, writing, a job spec. We're going to see a lot of really interesting things get built on top of image generation and models like Dall? E. And so I think Generative is a really, really important area. A second thing I'd call out is the combination of these large language models with program synthesis and the ability to take action on behalf of users, I think is something that we're beginning to see early signs of and we're going to see more and more of. So imagine you have models that can understand, but then actually act on behalf of the end user in an enterprise context, take actions in business applications to kind of automate workflows. I think that's going to be important. And then a third piece is just what's happening on the infrastructure side? So if you think about the pipeline to actually building a machine learning application, starting at data management and then doing things like model discovery, training, serving, monitoring, there has been a lot that's happened over the last 18 to 24 months, both in terms of new companies and open source projects that make each step of the pipeline much, much easier. And so for new application developers, new product builders, there's a lot to actually leverage and get your AI products to market much faster. And so, like, I think these are all ingredients that are going to lead to a complete explosion in the number of AI applications we see over the coming quarters and years.

You know, I'm struck, Saam, by your notion of this generativity and activity on behalf of the user that really seems like it's fundamentally different than what came before. It feels almost like to draw an analogy from earlier, the Transition from Web 1.0 to Web 2.0, things just open up and there's just so much more that's possible as a result.

Yeah, and one of the things that's interesting is in some ways it's almost easier when you have your product hat on. Right. Because a lot of these generative products, they can build in ways where a human's in the loop. Right. So I can say, you know, I can use a generative text model and say, hey, I'm trying to build a job description for a software engineer and the model can take me 90% of the way there, and then I can come in and do that last 10% of tweaking. And so, like, you know, when we think about backing new founders building an AI, we both think about, like, what's possible from a technology perspective, but then also what areas lend themselves to products that can actually get to the accuracy and effectiveness that end users need to really transform the way they work. And if you take that lens, I'm really excited about generative because I think it lends itself really nicely to these human in the loop workflows where the model can do the heavy lift, the end user and specialist can come in and do a little bit of fine tuning, if you will. It's an overloaded term here. And then get to something that's actually very, very performant.

Well, I'm very glad to hear that the human remains in the loop because us authors have a lot to fear if generativity just takes over. But I'm glad to hear I'm in the loop for just a little bit longer.

But this is actually an important point to kind of emphasize actually, in fact, people tend to jump too quickly, especially within Silicon Valley, go, okay, this, these products are on an improvement curve and so therefore soon humans will be out of the loop. That's not to say that it's not impossible in various contexts, but when you look at GPT3, when you look at Dall E, when you look at the music generators and all these things, if you actually say that, actually, in fact, you shift your paradigm some to thinking about these tools now become the essential tool that like, for example, you say, well, I'm a writer and, and if you kind of don't write using computers these days, you're actually very unproductive. I got my typewriter or I got my pen and paper. It's like, no, no, actually, in fact, the computer is useful in all kinds of ways. Well, this is like that amplification of that similar, like I'm a graphic designer or an artist. It's like, well, it's a tool that's like Figma or like Adobe, you know. And so it isn't that the generativity suddenly goes, now, Chris, instead of writing, you know, you can sit back and, you know, eat popcorn and watch the movie while, you know, GPT3 is writing for you. It becomes a thing that is an essential part of your tool set for how you generate quicker, better, faster, at a higher quality output for all of that. And it just becomes an essential tool. Just like we as creative professionals have these essential tools now. It's like, for example, you say, I'm going to create a movie, but I'm going to use it by snapping Kodak Polaroids. It's like, well, that doesn't work right, let alone not even as photography. And so these are just now the essential tools by which this is going to operate.

I remember at Greylock several years ago, we were discussing these things and there was a lot of skepticism in the kind of different experts and different people in AI we spoke with. And one of the things that I think everyone has changed their perspective on is that we're not hitting the asymptotes on scale, right? And so that's what's so interesting is you look at these models and as we increase the scale of data and training runs, they just keep getting better and better. And that combined with their ability and some of the kind of the mechanisms that have been unlocked, including the ability to train on different modalities of data, like again, really changes like the types of, to use Reed's parlance, magic they can do. And I think we're like very early days of that. I think if we're all talking again a year from now, we're going to be stunned by, by some of what's possible.

Well, that's one of the things about these technologies. I mean, people try to predict them and they always predict them in a linear way. Whereas actually we live in an exponential discontinuous world. And the stuff that comes is never the stuff that people predicted.

Part of the thing that I think is amazing about technology is that usually the progress is sooner and stranger than you think. So like in the 1950s, people thought flying cars because they had cars and they had airplanes, like okay, flying cars. Now we're just beginning to get the flying cars, you know, joby, other kinds of things as a way of doing this. But what do we get in the interim? Well, we got, you know, the Internet, we got mobile phones, we got a whole bunch of things which were never part of the Jetsons, you know, future. I mean, sure, Star Trek had, you know, communicators, you know, et cetera, et cetera. The smartphone's a whole lot more interesting than the communicator, you know, in Star Trek, even with a little like, you know, hit the little button on your vest as a way of doing it. The smartphone is more of an interesting platform. And that kind of sooner and stranger is the kind of thing that we'll be seeing that you're gesturing at with the kind of the exponential technologies or fast developing technologies. And part of what's interesting about AI is it actually will be amplifying a bunch of different human capabilities. And that's part of the thing is amplifying these capabilities in ways that it doesn't necessarily mean like everyone tends to go to the, oh wait, is that going to replace a whole bunch of jobs? And you know, for example, if you can provide, call it say a 2x amplifier in engineers, you're not going to shrink any engineering jobs whatsoever. Right? There is a such a massive amount of headroom for it. And by the way, that's not just true for that, that's actually, I think true for lots of different kinds of content creation. I think it's true for visual design and dall E. I think it's true in writing and other kinds of things. Like, well, if you give me an amplifying tool, sure. So that's part of where this kind of human amplifying, what some was earlier referring to as human in a loop, will be there for a fairly long time. Even as we're going to see this kind of magic of amplification and the kinds of things being done by these AI models that wasn't deemed as possible or near term possible, you know, even five years ago.

Now, it's funny you mentioned George Jetson. Of course, the Internet's gone crazy recently because it is, in fact true. If you look back over the cartoon, you can basically do the math. And George Jetson was born in 2022. So we're literally living in the beginning of the Jetson age. And what's funny about the Jetsons is it's this image of technology being just a part of the everyday. Right? Everyone has adopted every technology. Flying cars, briefcase cars, robots, you name it. Is that how consumer adoption of AI is going to proceed? Are people just going to dive into it and just start using dall E&GPT3 for everything? Or is there going to be reluctance? Is it going to be areas where people are like, I don't know, just.

Like any new technologies, like, for example, there are people who were slow adopters of smartphones, maybe even some people don't have smartphones now. You know, there's always the kind of older set, you know, the younger set takes to it like it's water, an oasis kind of in the desert and just does it. And it's much more digital native, et cetera. But on the sufficiently large ones, everyone ultimately starts and adopting. And that was part of the reason why, you know, when I kind of got a chance to start playing with Dall? E and looked at it and generated, you know, kind of some essays on it, generated some galleries, you know, did the NFTs I mentioned. But part of it was my very first day's reaction was a misconception, which is like, okay, everyone's going to have a B plus, A minus graphic designer in the pocket. That's actually a misconception because there's people who have no ability to generate visual. They can use Dall E. They can use it for generating a unique, you know, like holiday card, a birthday card, Valentine's Day, something that may be an image because they can use their language and words and kind of iterate through it. And it may not be as great or as artistic as a professional visual thinker and a professional tuner. And who knows what the whole. Who knows what the ways to make something really amazing are, but they'll be able to use it for a variety of their use cases. Emojis. Consider emojis in communications. And then on the ones who have some skill, like for example, you know, like a Graphic designer, an interaction designer, a illustrator, et cetera, who might be deployed in various contexts, not just even tech companies, but more broadly. Well, now they'll have a tool by which they can move more quickly iterate, you know, 30 different possibilities. And so, for example, you say, well, now they're, you know, when they're generating a logo for a small business where the previously they might have to charge a small business $500, now maybe they could charge a business $50. Because they say, okay, tell me what you want right now. And I start typing into, you know, Dall Escala. And I say, well, which of these do you. Do you most like? Oh, you want me to mod this one a little bit? Okay, I'll mod this one a little bit. Here you go. Right. And I can provide a bunch more. And then I think that goes all the way up to artists and, you know, like, where directors will do pre visualization of what scene they want in a movie. Or when you think about a large scale, like a public art large scale, you know, kind of insulation, you could start having Dolly generate kind of things and go, oh, this version of it, like. And again, doesn't mean you're going to do just the thing that Dall E generates, but it becomes a fast loop by. Suddenly you had 200 prototypes where you said, well, out of 352 and 78, those had the elements that I'm going to bring together in the thing that's even more amazing than the thing I could have done before. And I think this is true across all of these things. And so that gets to the consumer part or the everyday individual part is I do think that all of these tools can be more used by everyday individuals. Doesn't mean everyone's going to do it. Doesn't mean everyone has to do it right away. I think it'll become a professional standard. If you do it professionally, it'll be like, well, if I'm trying to do graphic design and I don't use Figma or I don't use Photoshop, then it'd be like, you know, I need to have those tools baseline. But, like, everyday individuals also create like a Valentine's card or something else as well.

You know, as Reid was going, I was thinking of what he was describing, which I'd put in the bucket of, like, new products and what will feel like new products and new experiences to consumers. And I think there's a lot that's gonna happen there. The other kind of framing that Reid and I often talk about is like, I was Thinking about the Jetsons analogy and like one thing that we've talked a lot about in Silicon Valley is this idea of like a natural language interface to of kind computing. In many ways the way we use computers is super clunky, right? Like, and I think, you know, because of some of these large language models, you will actually be able to just talk to your computer now. You may type, you may speak out loud, but you'll speak in natural language. We're seeing the early signs of that with how you generate images on Dolly or generate text with GPT3. But you'll have tools that will be able to do things like, hey, my mom's birthday is next Wednesday, can you please organize a gift? And you know, the model with the combination of actuation will come back and say, hey Chris, here are three things we know your mom likes. You know, pick the one you want. We'll use this API to actually order it for you, write a handwritten card and send it to her. And like, so in that way we'll have like this new interface to computing that will be quite interesting and I think people will adopt that. That's kind of one bucket. But then the second thing is AI is already like pervasive in a lot of the technologies we use and we just don't realize or many of us may not realize it. So you know, at Greylock, we're investors in ByteDance and you know, many of us have used TikTok and there are many things that make TikTok a great consumer experience, but one of them is just how good the relevance is. Right? Like I certainly find myself scrolling it for way too long because of how relevant the content it's showing to me is. And, and that's because of a lot of investment in AI to actually drive that real time relevance and model updating. And you know, we see that in the context of ByteDance. We see that in the context of something like DoorDash that you know, suggests to you a new pizza restaurant because it turns out you really like Napoleon pizza pizza. And so I think it's like we will see these new products. But I also think lots of consumers are and will in an increased way be interacting with these AI advances, you know, without fully realizing it.

And that's the thing, all these products, all these services, they're so much better than we imagined them in advance. And something like TikTok has this magical ability to draw my daughter in and just watch it endlessly. And you say, oh yeah, there's some silly videos. Why would anyone Watch that for an hour. And yet when you experience it, it's like, oh my gosh, there's no way to avoid it. The AI is so powerful. To which of course my response is, I for one welcome our new robot overlords.

Yes, that's the classic tongue in cheek Silicon Valley line. And I do think that one of the things that sahm's saying here is particularly important, which is there will be some amazing products that you'll be able to interface with and that's that generation of magic. But it's also going to re conceptualize a whole wide variety of services and there's already a bunch of services that are doing it. I mean, to some degree Search is powered by a bunch of different, you know, kind of AI techniques for kind of relevance and how that works. The news feeds and various things, you know, including TikTok and others use this as a, as an example. You know, there's already been for years an ability to have like upload a photo and then make the photo like impressionist or make the photo cubist or you know, do other kinds of things as ways of doing that. And that already exists within all these things. And they're all part of this new set of paradigm of AI tools. And so not only are they going to be new things, but everything you currently do is ultimately going to be powered and accelerated. And it even gets down to like the connection of atoms and bits because obviously the software transformation of the world, you know, kind of using a bunch of data and using AI is the kind of the exponentiating amplifier and all this stuff, well, that even begins to play to atoms, can play to atoms in like one of the things you already have happening is a bunch of different AI efforts led primarily by DeepMind. You're doing protein folding that will lead to all kinds of things that are possible. Evolution of MRNA vaccines or other kinds of precision medicine. That's one form of bits to atoms. Part of what's already happening with major pieces of machinery, including like planes and so forth, is you have a whole bunch of monitors on all parts of it. And it will actually do will monitor, is it safe, does it need preventative maintenance, et cetera, et cetera. How is that working? You've already had actually designs of like airfoils that come through kind of simulation and use of these AI techniques rather than purely just the physical thing of a, of a tube with a fan blowing through it is you use an iteration around simulation and kind of laws of physics in order to hone new interesting designs on it. And so the very infrastructure, the fabric of a bunch of things that we're already doing will be reinvented, renovated, revolutionized, exactly as saam's talking about, through a variety of AI techniques all the way into the world of atoms from bits. Now obviously BITS is where you start and BITS is where we tend to do, you know, where we anchor our work at Greylock. But it'll have a broad reaching set of transformations and improvements across society.

Absolutely. Now, thinking about investing, which is something that obviously is on your minds as partners at Greylock, you've partnered with a lot of these AI companies, but for every one that you've invested in, there's dozens or hundreds more that are out there looking to raise money. How do you decide where to focus and where are you seeing the most promising activity?

Well, both SAHM and I have made a number of investments and the two that I've worked on, one directly, one with sahm Inflection and Adept, both aren't super clear yet with the entire market as to what they're doing because they have some great ideas. In the inflection point, it's Mustafa Suleyman, former kind of co founder of DeepMind. At Adept, it's David Lawan from OpenAI and Google and a bunch of other great people from Google, you know, kind of doing this and some others. And they both have like a really good solid conception of building a unique product relative to the other things that we saw in the field. And it was one of the things that, you know, becomes a lens for looking at this because there's some stuff which is a standard, you know, like okay, do you have a good go to market? Do you have a really talented team? Do you have a sense of what competitive differentiation looks like? Do you have a well conceived business that might have a competitive moat or an ability to, once you establish a position is really good. On the other hand, you also have to understand this technology. It's a rare talent set. What's really possible with it? What are the things that are going to create some differentiation?

Yeah, I think to piggyback off of Reid, when we think about AI in the enterprise, there's roughly two buckets, applications and infrastructure. Maybe I can comment on each. So on the application side, it's early days. We think a whole new wave of SaaS and business applications is going to be built as AI native or applied AI companies. And the real idea here is take any function and can you leverage AI on top of data sets to really drive strong business roi? And so to give you examples, like on the cyber side, we think it's extremely fertile because in cyber you have so much telemetry and so much noise to sift through. And like the discriminative abilities of AI are actually really useful and kind of parsing the signal there. And so, you know, one example would be abnormal security, a company where investors are not Greylock, which is building a next generation email security platform. And they're, they're training on email data and organizational data and they build models of, okay, read. How does read typically behave on email versus how to som typically behave everything from language topics. And then when there's anomalies, they identify that for security teams and that's how they solve different types of attacks in the sales context. Companies like Cresta leveraging generative techniques. So at Cresta, kind of the mission is, how do we make every salesperson 100 times more effective? And so imagine software that's kind of your copilot as you're a sales rep at AT&T talking to a customer. Can we help generate suggestions for what you can say to the customer to help drive the business outcome you're trying to drive, whether that's a new sale or preventing a customer from turning. So I think there's a lot to do in applications where we're looking for applications where there's a strong connection between the model's output and business roi because then it makes it very easy to contextualize the power of AI to the customer. And then, you know, I'd add two additional things we look for, and they both relate to data. So one is data readiness and availability. So you have to train these models. And so we want to back products where they're entering domains or verticals where there is a lot of data that the customer has in a format that the product can actually train on. And then the second piece of data is like human feedback is very important. And so founders who can really craft these AI products that humans actually want to use interact with because that interaction ends up driving back into the model and making the model more performant over time. So that's applications. On the infrastructure side, it's all about enabling large enterprises to take better advantage of AI. And here I kind of think of three main thrusts. One is, if you think about that pipeline that we talked about earlier, Chris, from data to model and production, doing inference and monitoring for things like bias and explainability, each part of that pipeline is an opportunity. And so, you know, today we have investments in companies like Snorkel that Focus on data management and an automated approach to labeling data to make it ready for machine learning. Or companies like Truer on the other side of the pipeline, where when you have your model running, making loan decisions and you need to be able to explain and audit those decisions, Trara enables companies to do that. So that's one bucket of infrastructure. The second is actually empowering and making builders, data scientists and machine learning engineers more productive. So you know, you can think of like collaboration tools for these people, experiment tracking all sorts of things as this becomes a much more important user demographic inside large companies. And then the third is like democratizing AI. And what we mean by that is, you know, I think about the analogy of if you take tableau and lots of people inside companies know how to use tableau to get a sense of what's happened in the past in their data and what's the equivalent for what's going to happen in the future. And so if I have a business house who may not know what a hyper parameter is, but has a predictive problem, can we build tooling to actually enable those people to harness the power of AI? That's a big thrust for us as well.

Now speaking of the enterprise, it feels like there's a couple of different kinds of companies. There's these giants like Google and Microsoft, they have big in house AI efforts, they're trying to play in this space. But then other enterprises are looking to buy rather than build. What's the difference in how AI fits into the tech stacks of these different kinds of companies?

One of the things that actually I think is pretty interesting to look at as an example is what Microsoft and OpenAI are doing together. So on one part of it, Microsoft is trying to build a platform in the Juur using the kinds of products that OpenAI is doing to make sure that has the best compute fabric, the infrastructure for supercomputing and ways of doing it, enabling a set of what they call cognitive services in order to enable a bunch of AI generated applications within, within Azure. And on the other hand they're also releasing some first party products both with Microsoft and also having OpenAI do it as, as an exemplar. And there's kind of like three key things that I kind of point out. One is, which we've referred to somewhat already, which is GPT, which is currently GPT3, which is what's out from last year, but there will be a GPT4 that is coming and it has not only various forms of language generation, but also gives you translation and a bunch of other stuff basically for free. It's one of the things that's kind of part of the magic of these. One of the parts of it is you say, well, you train it on the Internet and text, but then you fine tune it on code and you have the Copilot product, which you know, last year's model with you know, 100,000 developers accepting 35% of the coding suggestions. And these aren't like, you know, finish the variable name. It's a, oh, you're trying to do this kind of activity that you want to import this library, use this library, write this code, whether it's a sort a data processing, you know, any kind of algorithm, any kinds of things and doing that sort of thing. And then Dall? E, which we've already mentioned, which is kind of doing image generation. And part of the reason why I went out and did these kind of essays on how important Dall? E will be is I actually think that it'll be, there's going to be a landmark here of the world of graphic design. That's pre Dall? E and post Dall? E just kind of like pre Adobe Photoshop, post Adobe Photoshop, pre Figma, post Figma as a way of doing this. And I think that's all part of the kinds of things that will be coming out of as also first party applications in addition to third party party.

Fabric and maybe to add to, to that lens from the enterprise perspective. You know, we think about this bifurcation, Chris, as you alluded to around build versus buy and, and our mental model for it is build is when a company is really building something that's their core competency. Right? So like you could take the most extreme view and say Google Search, right? Like Google Search. Every, every additional point of accuracy and effectiveness is critical in the core competency that business. And so that's not really an opportunity for third party vendors to come in and sell tooling against it. And, and you know, that's a very extreme example. But you take banks for example, banks who are underwriting credit products. These are areas where these companies are investing in their own internal data science teams. They want to eke out every additional point of performance. There are some opportunities at the edges of that to sell highly specialized infrastructure, but we've stayed away from that because we're not. It's a more niche opportunity. On the buy side though, the lens we use is where are there horizontal business workflows that are important but not the core competency of any one organization. Right. So document processing and Extraction, contact center, AI. There are these core primitives that are shared by how these enterprises operate and are important in great application areas for AI, but not core to any single customer. And that's where there's really an opportunity for these third party applications to come in and sell and companies will buy and that's where we look to invest.

It sounds like when you talk about infrastructure, it's almost like a classic platform business model. Now it's all of a sudden you're enabling so much more and all the people who build on top of your primitives are increasing their value.

That's right. That's exactly right.

One of the reasons why AI is super interesting and one of the things that Sam and I and others at Greylock focus on also, of course, why we also paying attention to Web3 is new technologies become new platforms for either the reinvention of existing applications or the generation of new applications. And part of the reason why we've been kind of going through, you know, not just the discriminatory models, but the generative models on this is the number of kind of new kinds of applications that just simply weren't there before now. And everyone naturally tends to go consumer in their thinking because that's the more broad. But even the enterprise is as magical and as amazingly interesting.

Now, speaking of that consumer side, you know, I'm a creator, I have a degree in design from Stanford, so I did a bunch of studio art. I have a degree in creative writing and obviously I'm still an author. And so I'm really excited about things like GPT3 and Dall E. So these are just astonishing, right? They've left behind these crude and primitive, hey, this is a cat. Cat. Or a chat bot for some sort of customer service. How did we get here? How do we get to the point where we're actually creating real content with AI?

By the way, most experts didn't actually predict that the large language models of foundational models would get as magical as they would. Most of the pieces of technology have existed for several decades, but what happened is when you threw enough compute at it and you threw enough data at it, and you add in some other mechanisms, for example, how to really use all that data, you know, whether it's the transformer model or, or the self play model, that all of a sudden on just intensive amounts of compute over the data, you end up with magical applications. And so for example, one of the things that what happens with GPT3 is you can, you can feed it things like prize essay questions from the All Souls Prize Fellowship in Oxford University. And it says, there is no Marx without Lenin discuss or Schrodinger's cat challenges our concept of reality. You know, true, false, and it will generate something that's actually in fact human and coherent and interesting. Doesn't mean that it's always an A plus essay. Matter of fact, actually, you know, sometimes it's kind of a B plus essay, but. But even B plus is amazing. Was. Wasn't actually, in fact doable for. And it comes from. From the fact that it's this huge model that's been trained on generativity relative to what's interesting. And you say, well, there's at least anecdotal parallels to what human beings do, right? Because you kind of say, how do you teach writing? And so it's like, well, okay, oh, see, spot, run. And then it's like you start going, okay, what's interesting? And people say, oh, this is a great use of language. This is a great way. Yeah, you've now spoken, you've now written the right way. And then we generate. Now, whether or not it's anywhere close to the same mechanisms that we use to be generative, there's some dispute on. I tend to think it's inspired by the way we do it, but it's actually, in fact, very different among them. Kind of in the more kind of geeky thing is part of what these large language models do is they deploy huge amounts of ability of scale and electricity in order to do it on a kind of cognitive capability, as you were like an IQ point per watt, our brains still function a lot better per unit of electricity. Were much more generative and so forth. It's just that this other architecture allows you to apply scale in a really interesting way, which allows you to generate things that you previously couldn't generate.

That's one of those fascinating things, right? Efficiency is great, but scale has its own power, as we've discussed in the past.

Nice little bow, Chris, to our work in blitzscaling and everything else, which I saw you going there.

I was tearing it up for you. Now, obviously, we're getting pretty excited about this, and that's because to be in this industry, you kind of have to be an optimist, a techno optimist, if you will. But there's also a dark side to these things, right? There's a lot of ethical considerations to take into account when developing AI. So what are some of these top ethical concerns in the field today?

It's a really important area. So on the enterprise side, there's like two buckets of challenges. One is, how do you know that the data you're training your model on is actually representative of the real world and also preserves the underlying privacy of whatever you're training on. And so, like, I think that's a really important area. And we see different tools and projects that have emerged that enable you to understand distributions of data sets, enable you to do things like create synthetic forms of your data set so that actually a data scientist can go in and play with the data and look at it, but not actually compromise any of the underlying privacy of the data. And I think that's actually really important to take advantage of these data sets that live in the enterprise and have such strong predictive powers. Then on the actual inference side, the key thing is actually being able to explain why a model is making a particular prediction. And then once you have that fundamental explanation, you want to apply different lenses to it. Like the canonical example I think about is the loan approve or disapprove example, right? So you have a model, you feed it some data about a borrower, and the model comes back and says, this loan's been been denied. Now there are a list of important regulations actually dictate the kinds of things that you can deny a loan for that are kosher versus not. And then the question is, did the model adhere to that? And so in order to answer that, you need to actually be able to explain why did the model deny what were the different variables and parameters around the data vector, if you will, the applicant that led to that decision? And is that fair, right? Or is that ethical? Might be another way to approach that. And so I actually think a lot of the real world deployment of AI in areas where it could be very useful is rightfully blocked because we don't yet like as an industry have what the answer here fully is. And what we have to get to is an agreed upon definition of explainability, agreed upon standards on how we actually audit these decisions, and then a way to apply, you know, an agreed upon set of standards from a fairness and bias perspective on top of these explanations. And we're still like, lots of good work happening, but still early days there.

Plus one, everything SAHM said and then expanding some. One part of this is that currently AI is very difficult to actually look under the hood and know what's going on. Unlike when you write an if then kind of code, like, you know, if a person is, you know, 40 years or older and has a FICO score of X, then blah, blah, blah, then you go, okay, I know how it's operating. These large models are intensely pattern coordinators, but they're very hard to see exactly what they're doing. And so you say, well, is it being fair? Is it being just? Is it operating when it's unusual circumstances, how is it operating? And this gets to another area that we've been investing in a Greylock, which is autonomous vehicles, things like Aurora, things like Neuro, things like Nauto. You know, there's also Uber for trucks like Convoy. But you look at these areas and you say, okay, well, what is the AI and safety implications here? It's like, well, you really want to make sure that it's great relative to human life and relative to creating traffic jams on highways and trucks and so forth. Now, one of the unfortunate things is human beings tend to want to do this as perfect safety. And you're like, well, actually, in fact, about 40,000 people a year die in the U.S. you know, in car accidents today. And by the way, part of that, like the data from Nauto, and so it shows that a huge portion of that is the 10% of worst drivers. So your standard of, if you could actually get to, like, everyone to the, just the, the, the middle point and the autonomous vehicles got everyone to the middle point, you'd save tens of thousands of lives, everything. Even if you had still, you know, 3,000 deaths or 5,000 deaths per. So you have to balance out what that safety is. It's really important to have safety premium. If you said, well, this one is eight times safer and going to save seven lives for everyone who does, that would be something that we should want. Now, of course, part of what gets funny about this is that people say, you ask people, you say, well, what would you like people to do? And it's like, oh, I'd like everyone else to be in an autonomous vehicle car that, that emphasizes the maximum safety of life, and I'd like the one I'm in to be safety of my life, right? And you're like, okay, so you have to work out these kinds of social issues. Obviously, there's kind of the unintended consequences. And this is part of the reason why, you know, most Hollywood films tend to go, the robots are coming for you. And actually, in fact, you know, I actually think frequently question is, can the robots get here soon enough, can get here soon enough to help us with our elderly care problem? Can they help us with our new amplifiers of human manufacturing? Can they help us with dangerous cleanup areas, whether it's, you know, wildfires or, you know, other kinds of things in terms of how to have things happen. But you still have to look at, you know, are there unintended cyber consequences? So if you said, well, we have a good, really good cyber defense thing, but what if the cyber defense thing breaks or, or somehow breaks the network in some way? Well, that would be, that would be an important issue. And so you have to look at, you know, what frequently is referred to as the paperclip problem, which is it's maximizing the wrong function and that has very bad side consequences. You have to look at all of this stuff. Now one of the things I think that people don't realize is that it's again, just like the autonomous vehicles, which is most people who are doing this stuff are thinking about safety stuff, are trying to do it doesn't mean it's perfect. And what we're trying to do is figure out, well, what's the best safety we can get while still getting all the improvement. And so like I wrote an essay, oh gosh, Money, even almost 10 years ago now that was saying, you know, we should want the autonomous vehicles to be here as soon as possible because, you know, today's autonomous vehicle technology from Neuro and Aurora and others would actually in fact already be saving tons of lives if it was deployed. Obviously there's a bunch of work to still do. But anyway, so those, those are a whole bunch of ways of thinking about the ethical considerations and the fact that people are actually in fact engaged in it, thinking about it, talking about it, trying to work on it, still a bunch of work to be done.

In addition to AI, spent a lot of time on security at Greylock and I'd be remiss not to add on like the security angle here. So I think there are, there are a number of potential threat vectors and opportunities for adversarial attacks on machine learning systems. So, and that's an area that like we're paying a lot of attention to and we think there's opportunity to strengthen and like two of the things that for example, we think about, one is on the data poisoning side, right? So if you think about these models, they're entirely informed by the data on which they're trained, right? And so if an attacker comes in and actually poisons that data set, you can create a model that's now making predictions that are anchored in a really adverse way. And so that's one big problem in an area where there's a lot of work going on and we've looked at projects. Another area is like on the what's called model extraction side, which is you have a black box machine learning system that was trained on some data, and let's say there's sensitivity around that data, or there's privacy issues around that data. And now an attacker comes in and keeps hitting the model for different inferences with different inference requests, with the goal of backing out some of the underlying semantics of the data the model was trained on. Right. And so, like I just described, two attack vectors, there are many. But as these systems become more ubiquitous, the surface area of risk also increases. And so it's like an area we as an industry have to spend a lot of time thinking about. And we at Greylock think about and, and meet with entrepreneurs who are trying to kind of solve these problems as well.

You know, as I hear this, I'm taking heart from it because it sounds like people are really thinking about very carefully. But I also can't help but picture the next Terminator movie with octogenarian Arnold Schwarzenegger, where he's working in an elder care home and he's telling them, come with me if you want to live a longer and healthier life. That's what I'm picturing now. So one final question then, which is to take a look into the future. As AI continues to evolve, what innovations do you two hope to see in the near and far future?

Well, one of the things that I think is really fun about the whole venture business is while we work on our prepared theses, we talk with great entrepreneurs, we talk amongst ourselves. We have a bunch of things. It's obviously being surprised by the thing that we don't see that the really amazing. There's a little bit like the sooner and stranger. And that's one of the delights of the job, is finding those entrepreneurs where she or he might be like, I've got this idea and vision. You're like, wow, I never really thought about that. It was kind of like the, you know, when Brian, Joe and Nate were talking to me about Airbnb, you know, my first investment for Greylock. And I think that's the kind of thing that, like, as we hope to see, is we hope to see that amazing entrepreneur coming with something. And the reason why I open that, because that's the most important thing I think you do as an investor, is have a prepared and open mind for finding the transformative idea, the new idea that's out there in the world. And those are the kinds of technologists, inventors and entrepreneurs. But I do think that what you're going to see is the set of things where you begin to say, well, what are the really major areas that matter to human beings? And it's like, well, medicine matters. I think you'll see a bunch of stuff that'll help within kind of precision medicine and other kinds of things. And that I think education will be similar. I think there'll be a bunch of things that lead to like, okay, how do you optimize energy in the grid? I think there will even be impacts on climate change relative to maybe implications of how do you model a fusion environment or fission environment and make that a lot better. These are just all some of the things that I think you begin to possibly see. And none of that even begins to approach the infrastructure of which our society works, which is part of what makes actually in fact, enterprise investing so interesting.

Yeah.

And I'd second what Reid says, which is, I think the privilege of what we get to do is we get to meet entrepreneurs who are building that future and we dream alongside them. So we look to them to, to navigate us. That said, there's a lot we're excited about. I mean, on the enterprise side, there's a lot of like, narrow applications that I think are really important. So, you know, Reed mentioned medicine. Like on the software side, you think about the electronic health record systems that these hospitals operate on and like when you go in and see your physician and you know, they're writing down notes on the encounter and then they're kicking off a bunch of workflows based on, you know, what you need coming out of that encounter, like all of that. We're investors in a company called Notable Health that's kind of automating that end to end workflow. Like there's a lot of these kind of pick your vertical, pick your function, and AI should make the process much better. And I think that's going to pervade everything over the coming years. But the other thing I'd note is automation is an area we've just talked about a lot. And there's these like robotic process automation companies that have come out and different forms of kind of automating these discrete workflows. But to date, like, it's all been brittle and narrow and very kind of workflow specific. And I think the combination of large language models, some of the advancements in actuation and program synthesis, the ability to operate across modalities of data, I feel like we have the now core technical building blocks where we can really deliver on this vision of how do we make the knowledge worker 100 times as productive and how do all three of us show up at our desk and have something on our computer that's observing what we're doing, saying, hey Reid, you know, I'm noticing you're doing this thing over and over again. Can I help you do it? Hey, but you know, then it starts doing it. It makes a mistake with natural language, read can correct it. It's now learned the next time it doesn't make the mistake and now reads 10 times as effective because he has this like true AI, you know, assistant or copilot. I think that is something we're going to see happen on the bit side and on the enterprise side or investors in a company which is using models for prediction of RNA structure to design a whole new class of RNA targeting therapeutics that could be, you know, transformative to a bunch of, bunch of different areas of pharma. Reid mentioned Aurora Neuro, like the advancements there and you know, how that's going to transform not just self driving, but the derivative impacts of that across the entire supply chain. Like, I think it's going to be very, very impactful and I'm really excited to see what the next generation of entrepreneurs and great companies come up with.

Well, that brings us to the end of today's discussion. A lot of fascinating topics we just scratched the surface and I'm going to be very excited to hear you guys dig deeper into all of them with your guests in the coming weeks. Reid Sahm, thank you so much for joining us today.

Chris, thank you as always.

Thanks, Chris.

And for our listeners out there, you can subscribe to Gray Matter wherever you get your podcasts and you can find all our content on our website, greylock.com you can also follow us on Twitter reylockvc. I'm Chris Yeh and thank you for listening.