Jerry Chen | The New New Moats

Hi everyone. Welcome to Gray Matter, the podcast from Greylock. I'm Heather Mack and I'm here today with Greylock general partner Jerry Chen. Six years ago, Jerry published the new why Systems of Intelligence Are the Next Defensible Business Model. In that essay, he postulated that startups would be able to build defensible moats using AI. In light of all the developments in the past year, he's revisiting this framework to see what still holds true and what has changed. This analysis can be found in his new essay, the New New Moats. In this essay, Jerry essentially took a red pen to the original essay. He updated and amended predictions, posed new questions, and threw out the ones that didn't hold up. Jerry is going to walk us through some of the key takeaways from this essay. Thanks for being here today, Jerry.

Heather, thanks for having me. It's fun to always talk to you about our latest writings. In this case, it's fun to kind of revisit something we worked on six years ago.

So obviously AI is having a big moment right now and that's playing out in a lot of different ways. Before we get into the specifics with the essay, what was the overall question you were trying to answer with this analysis?

I think going back to the first blog, the New Moat Systems Intelligence, either the question was asking, debating with was, you know, where can startups build essential value, especially in the face of Big Cloud, that it seemed like every start we're looking at was trying to build something against Amazon, Google and Azure and trying to find some kind of go to market advantage, some network effect, manage advantage, et cetera. It was really a bunch of conversations around that topic six years ago led to the new moats and this theme around machine learning. AI was really in earlier wave of innovation and we kind of postulated that the systems of intelligence incorporating AI ML into your app as a way to build new moats in the face of Big Cloud. And obviously you and I worked on something called Castles in the Cloud, kind of a follow up project to the new moats around understanding where the vulnerabilities around Big Cloud existed. So that kind of sets the backdrop of where the modes where and how can startups attack Big Cloud? And it kind of comes full circle that six years later you want to revisit this thesis of systems intelligence in light of all the technology changes the past 2, 3, 4 years and how.

Would you characterize the way AI is being used right now?

So I think how we interact with AI has changed dramatically from kind of a Very stilted. You know, Apple, Siri kind of got it right, kind of didn't get it right to holy cow. GPT and Bard eerily know what I want to talk about or can kind of answer my thoughts or questions when I kind of ask the kind of very abstract concepts. I think there's two ways AI is being used by applications. Number two, under the hood, these large language models that kind of power these chat bots like ChatGPT, this would be GPT from OpenAI or Palm from Google. These large language models are actually pretty game changing in terms of really problem solving, reasoning engines. They actually solve problems and help us kind of be more productive. So either one, change how you interact with the world or number two, use these large language models to actually build something net new that's pretty powerful.

So would you say that AI is an option for companies to use or is it becoming something that is essential to even running a business?

Yeah, I think we were kind of joking around AI or DAI because it had this kind of nice little rhyme. But it's more like evolution of what companies and technologies are. These are the great waves we're seeing. So I think in the early phase of any kind of technology wave or new platform shift, you're like, yes, it's a mobile app or mobile first, or it's web first, web only, or it's cloud first, cloud only. I think we're the early days of saying everything's AI and really what we mean is like AI is going to permeate enterprise software, security software, consumer experiences, your self driving cars, or home automation, your healthcare. So in some small way, just like the Internet is part of all tech today, like mobile is part of all tech today, AI will be an ingredient in some small or large fashion in all applications going forward.

What do you mean by systems of intelligence when you're talking about it from six years ago to what it means today?

So if you look at the enterprise stack or just company stacks, historically we've always talked about systems of record, right? A system of record is your CRM, your hr, your IT database, your customer database, your employee database. There are handfuls of systems of records out there that are super powerful modes because they are, no pun intended, the system record, the data record of all your customers. That's CRM. That's why Salesforce is such a powerful company. We also talked about systems of engagement. Systems of engagement is how you interact with their applications or data. And that's everything from a web browser, a mobile app, operating system. Increasingly we, we talked about Six years ago. How Slack? That was pretty popular back then. WeChat, WhatsApp, how chat metaphors or voice metaphors like Siri were going to become a new system engagement. And so you typically had the system engagement running on top of system record. And both areas were markets for startups compete, most notably in the browser wars. When we thought the web browser was a system engagement, you had Netscape versus Microsoft Explorer in the system record. Clearly we've had large companies from ServiceNow, Workday, Salesforce, become these kind of de facto databases for companies, in addition to the actual database of companies like Oracle, et cetera. And so in the past we looked at these two layers as areas of build modes and build differential business models. We introduced this concept of systems of intelligence six years ago, saying, okay, if you're a startup, it's very, very hard to own a system of record. That usually only happens when you have a platform shift from going from on premise to cloud. So you went from Siebel to Salesforce. So in lieu of a platform shift, can you either A, build a new system engagement, B, build a system of intelligence, or C, attack the system record. And we're seeing that across the board. But system intelligence is using AI, machine learning or other technologies to kind of bridge across multiple system record. So how can I use machine learning in AI to look at my HR data, my customer data, my employee usage of that data, my customer behavior, and create a net new workflow, a net new application that kind of looks across some records and make intelligence recommendations or intelligent decisions on top of that data. In addition, because the AI is so powerful, it can turbocharge these systems of engagement, delivering information to you when you need it, where you need it. And so it was kind of an early thesis and we've seen a bunch of startups in the past six, seven, eight years kind of build around system intelligence, attack with system engagement and try to displace a system record. And it's been kind of fun as investor and to work with a bunch of founders in this category.

And so what is the critical component that's supercharging these systems of intelligence today?

So if you look at what's happened the past six years, we got right and got wrong is we do not foresee these large language models, right? So Transformers, the Transformer model, as technology really kind of revolutionized how we think about building large, large models and these large language models with trillions and trillions of parameters have become eerily like problem solving reasoning engines where they can not only understand our questions, their Answers can predict kind of the next word, the next token, the next image, et cetera, of what we want back. And so what's changed in the past six years are these large language models have become a cornerstone ingredient to help problem solve and help kind of bridge all the system records and build these systems of intelligence. And that's kind of the most exciting thing and that's shown by all the excitement around OpenAI and GPT. Google obviously has done a lot of this work and more recently released Bard on top of Palm. There's a bunch of open source models out there. Facebook released Llama, and from Llama a bunch of new models have been built hugging face and the whole open source community have been building their own large language models. So it's been amazing to watch both the rate of innovation in the academic world, but also in the private company world of these, these large foundation models.

And so we have these large language models that we have all these chat functions that are essentially serving like as a front door as to everything now. But how do we get there? What are all the different ways that companies are making it possible to leverage these large language models?

Well, that's actually a great question. And it's, we're looking kind of this new AI application stack that's being built. So if you looked in the past, you had these systems, engagement via chat, mobile, et cetera, you had the systems of record and you had a bunch of enterprise data sitting out there from databases, applications like notion, Coda, Tome, PowerPoint, et cetera, Google Sheets, the big SaaS apps we've mentioned like ServiceNow, Workday, Salesforce. What companies are now looking at building is how do I connect and bridge all that information sitting in a Dropbox folder to a foundation model and then have my end users or customers interact with it. And so we're pretty excited that we think there's a new AI infrastructure stack being built. There's companies like Llama Index that we invested in. It's kind of this data framework that helps bridge your private data, your personal data, enterprise data, with foundation models to let foundation models kind of interact with all your data, actually be super useful. Llama Index and other frameworks like LangChain help build agents. These autonomous agents let you do amazing things behind the scenes like interact with data, actually take actions on business process flows or personal process flows. The whole use of vector embeddings, quote unquote, the memory for these large lams models we're watching. So all the vector databases out there, we're trying to understand how that evolves into an independent category or do current databases like postgres, Elastic, Redis and other things like Rockset that have vector embeddings just become a vector embeddings plus SQL query database. And so I think we're still kind of understanding, you know, how the stack evolves. But as an investor in cloud software and infrastructure software is kind of pretty exciting to see these new frameworks take fruit and grow.

And you said you're still kind of trying to figure out where this is going to go. What are the risks for startups that are taking this approach?

You know, I think the risk for the startups take the approach is one, that underlying technologies are moving so fast, especially around memory and retrieval. Number two, the shift of models between big players like OpenAI and open source is still TBD and up in the air. And so I think the role of these open source models is still in question. I think there's a debate out there, Heather, around big models versus small models. You know, the world will exist with both. But do you primarily orbit around, you know, GPT or do you see a world with smaller models? Make sense? And I think finally, you know, this new stack is early and evolving. And so I think as developers build around it, you know, making sure they're very thoughtful and very agile around changing these things is going to be very important.

Right. And speaking of open source, you'd initially written that open source actually made it harder to monetize technology advances. You predicted that open source would only favor the biggest cloud providers. Do you think that's happening?

Yeah, you know, I think we overestimated and probably over rotated Atheist Heather, six years ago, I think the role of open source, one, we believed it was always not just how you create value or create distribution, but also where in the stack does value flow. In the past you created an open source project that helped move value away from that layer to adjacent layers. And for example, there's a bunch of open source technologies out there, from operating systems to databases to JavaScript frameworks, and very few people monetize React. But people can build amazing applications using React. So the framework itself isn't a company, but tons of companies are built on a JavaScript framework. And we argue that those that had scale and distribution could actually take advantage of open source better than a lot of startups. And that was big cloud. So what Amazon's done in terms of taking open source projects, running them on their cloud has created billions and billions of dollars of value. But we did argue in the follow up blog Castles in the cloud where we talk about open source versus the big cloud. You know, Amazon, Google, Azure are definitely monetizing a lot of open source projects, but open source is still a super valuable, super viable way to attack the cloud providers because one, you have better distribution. So to make up for your lack of a sales team that Amazon, Google or Azure may have, open source developer communities actually can help you with go to market number two, it actually lets you own the audience. You are the company around MongoDB, you, you're the company around Elastic, et cetera. So it felt like open source actually help startups win and create a niche around itself. That was kind of the great debate. I called it a rock and a hard place, open source and a cloud place. And startups can navigate that channel between rock and hard cloud and open. But it wasn't easy. What's changed now is, or what's in addition to that thesis now is open source foundation models. So if you took a step back and said holy cow, these foundation models would take billions and billions, hundreds of billions of dollars to train, billions of dollars to run and execute an inference and you can get access to these Nvidia GPUs to train them. How in the world can a startup compete? And that would normally favor big cloud. And that's probably a reasonable bet. But I think open source has kind of shifted the availability of these large language models, right? The Jama Vicuna Llama and the brethren. All of a sudden startups can now grab a version of open source large language models that's almost as good in many ways, in some areas better than some of the big models from OpenAI and Google and actually build their own system intelligence. So I would say yes, there's a battle of a who can make money and monetize foundation models. Is it going to be the big guys or the startups like Adept Inflection, cohere anthropic, can the small startups actually monetize foundation model? That's question number one. That's a battle beef all today. But I think question number two is in addition to the foundation models, where in the stack do you make money? And the existence of open source models allows companies like Tome Coda Notion Abnormal Security to grab technologies and build applications. Like, you know, Harvey AI is kind of a great example of an illegal vertical. So you can either use GPT4 from OpenAI or some startups and grab their own open source models and build it. Instead of fighting at one layer, they move the value to the JSON layer, the application layer. So I would say take a step back, rock in a hard place. Cloud and open source place still holds true. Number one, I think it's Number two, there's a great way to use open source to make money against big cloud. But I think number three is the question is really is where in the stack does value accrue? Is it going to be at a foundation model layer, is it going to be at a higher layer in the infrastructure layer, or is it going to be at the application layer or someplace else we haven't talked about? And that really is the question of what is a system intelligence and why. We believe that could be a thesis around building sustainable value.

Right now there's much more detail in the essay, there's a lot more case studies, a lot more reference points. But to recap here, it's baseline incorporate AI or die, to speak a little harshly. And then we're on to chat is shaping up to be the front door to essentially everything, or at least one of them. Foundation models have become the cornerstone of the strongest systems of intelligence and getting there utilizing those systems of intelligence requires a new software stack, which we're seeing happen from all these different startups you just mentioned. Open source AI has actually spread value between startups and incumbents. So what does this all tell us about where today's moats are?

I think what it tells us it's a great time to be a founder and an interesting time to be an investor trying to kind of parse through this landscape. One, I think there's definitely an opportunity to build a moat around system intelligence. I think that story, that theory we possibly 6 years ago still holds true. So I would say if I'm a startup founder, definitely think about working build a system intelligence. Number two, I think around the core foundation models, definitely open source or not open source, there's still ways to beat big cloud, right? I think the technology is changing, the open source community is fast moving. So we're excited to watch how and when and where kind of these startup foundation models win and beat the incumbents. And then number three, like I said, there's this idea of chat or these AI agents being a way to enact with the world. And I think you're going to see some startups be the equivalent of a full stack application. They're going to build a model, build the infrastructure, build these like personal AI chat bots assistance for us and build the whole solution. We're going to see some startups try to just build a model down the ground, build the app on Top, like in the legal vertical or some startups kind of build something in the middle. Either be the infrastructure plumbing to make this happen or to build systems of intelligence, the next new defensible business model.

But at the end of the day, you know, no matter how the technology underpinning each generation is, however that looks the fundamentals of business building still remain the same. Right?

Yeah. I actually think one of the things we learned the past few years is if AI is everywhere. So everything's an AI app, just like everything's a mobile app. Having an app on the iPhone or Android isn't emote anymore. Right. It's the early days. If you were the first or early photo sharing app on the iPhone, like Instagram was and had a great network effect, virality, you got ahead of steam, behind you going to the market. And so all of a sudden what happens now is if AI is ubiquitous and you know, within, if AI is now an API call away from every developer, if everyone is using some large language model and that's kind of an ingredient, then the old modes really are the new modes. Right. The way to build a financial business model is, you know, kinds of scale, network effects, switching costs, integration with other products, other technologies, better workflows, better user experience, good brand, good customer trust, better go to market advantages. So I would say it may be after six, seven years, Heather, we realized that there were no new modes and that the thing we should really care about as both founders and investors is yeah, all the fundamentals of building good business, selling a product, partnering with the ecosystem, still hold true. And that's probably why investors and founders have been talking about these moats, like network effects for years and years and years. And they will still be talking about these moats for years to come.

Got it. So things are the same, they just look drastically different.

Things are the same, they're just in a different blog format.

Definitely. Awesome. Well, Gerry, this has been really fun. I encourage everybody to check out the essay, which is now available on greylock.com, we've also shared it on all of our social channels. And we want this to be a conversation, so we invite everybody's feedback. We'll be having a series of conversations with some people in the industry in the next coming weeks and months. And for now, thank you so much for your time.

Thanks, Heather. Thanks everybody.