Launchable: Foundation Models - Matt McIlwain on "The Intelligent & Generative Apps Phenomena"

All right, I'm excited to transition to Matt. So I'm very proud to introduce you to our first speaker, really the perfect person to set the stage for this event, Matt McIlwain, the managing director of Madrona. Matt has been a full-stack VC throughout his decades in venture capital, working with founders truly from day one and for the long run, companies like Appdew and Smartsheet, where his early commitment and support were instrumental in the company's eventual IPOs. Matt built early conviction in the intelligent applications theme, leading several investments in the category, and has a unique perspective on where the market is heading as foundation models take center stage. I'm grateful to call Matt a friend and mentor and excited for him to officially kick off our Launchable speaker series. Thanks and welcome, Matt. Sorry about that.

Well, Mike, thanks so much. And I think I'm gonna try to share some slides here. If they'll let me do that. Okay, great. Let's do that. Okay. Can everybody see those? Yep. Great. And let me just put them into the order we want them. Well, again, Mike, thanks to you. Thanks to the entire team. It's really a treat to be with everybody. I've been at Moderna for 23 years, and it was just in the transition from all the absorption of the early hype and excitement of the internet to the trends around software becoming software as a service back then. And then about 15 years ago, this thing called the cloud came along. We hosted an event with our friends from Amazon up on Capitol Hill here in Seattle to launch AWS. And then not too long after that, in 2012, 2013, we started realizing that the opportunity for machine learning, machine learning models embedded into applications, what we came to call these intelligent apps, were likely going to be a very, very big thing. And so as Mike says, we've been around the intelligent app world and now the generative app world for the last decade. And I loved this choice of title, the phenomena. Phenomena can come and go. But our sense is that this is the biggest thing in terms of disruption, innovation, and opportunity since the coming of the cloud and the mobile era that coexisted together. So I want to take just 15, 20 minutes or so here to frame a little bit some terms and some phrases that we use and some companies and some really interesting questions that we're asking and perhaps you're all asking too. Whether it's in the context of starting your own business or trying to figure out how to leverage this generative foundation models and generative apps phenomenon. And so I'm going to take you back, in this case, only, believe it or not, 7 years. So this was a framework. I'm still a recovering consultant, even though I've been a venture capitalist for 23 years. This was a framework that we put together in February of 2016, so almost exactly 7 years ago. And it was the first time we started calling applications intelligent apps. I'll define that on the next slide. But we were really trying to get our head around— you see over here the data that was being inputted into enabling infrastructure. That could be on-premise, and often it was as people were training their models, or it could be in the cloud. And then there was this idea that on the enabling infrastructure, you'd work your way up. You'd apply some of that algorithmic approach. And you would ultimately embed it into an application or a service. And then the last concept that was really important— in fact, we even called this the dataware framework because it really wasn't software anymore— was this idea that there would be this continuous feedback loop. And that would be the way that you would continually improve the underlying models and therefore continually improve the quality of the inference, the predictions, the recommendation systems that you were building. So that led to a definition. And that definition, with a little bit of the highlighting on the left, is an intelligent application being something that is a continuous learning system that solves problems in some contextually relevant way. You got to give it context to be able to solve the problem. There's a lot of different ways you can do that within the model itself, as well as within how you're using the application. We're going to get to a little in just a minute here, this big leap that's become the generative leap. But this intelligent application construct is a continuous learning system. These systems are based on a set of foundational things. Yes, foundation models is a part of that, as you can see here. But there's a whole ecosystem of data and data tools and infrastructure tools, of course, massive amount of compute power and GPU power that you need to train the models. But I think as we're also learning and we're hearing reports about, there's also a massive amount of compute to run those models, to do the inference side of those models. I think that one of the big questions is, how do I optimize the foundational inputs, the enablers as it were, so that I can cost-effectively in an economically viable way, actually run my intelligent apps, whatever those intelligent apps might be. And then you can move up to the top here and just highlighting this notion that these applications are going to be iterative in what they're being able to produce. And I'm going to show you a whole bunch of examples in just a second. So why don't we get to that to make it a little bit more tangible for all of us. So of course, a decade ago, we were already seeing— I mean, the Netflix competition on machine learning was well more than a decade ago. And, you know, we've seen in our consumer lives, whether it's search, which in and of itself is a form of intelligent application, or utilities that have come along like Alexa, and then some of these different, you know, applications that we enjoy different forms of content on, you know, Spotify, Netflix, and then our iPhone itself, you know, interestingly enough, in 2016, one of the companies where I learned a lot about what was possible with intelligent apps was a company we invested in called Turi, T-U-R-I, originally called GraphLab. And that company got bought by Apple, and then Apple turned it into the platform for all kinds of intelligent applications that were built inside the iPhone, inside your AirPods, inside your watch, et cetera, et cetera. And of course, Microsoft, as we've all experienced, has increasingly been embedding. But what was changing in the last several years was this whole idea of enterprise intelligent apps. And you note that there's a number of companies here in the consumer space. There's also these companies on the enterprise side, companies on the sales and go-to-market enablement area like Clary, like Gong, like Highspot, some very early adopters of leveraging data, differentiated data sources, and then machine learning to solve some problem better than it's ever been solved before. And now you've got this generation—

Matt, sorry to interrupt you. Just to give you an FYI, you're in presenter mode. See if the, the rest of the group can see this clearly. There's some great content here, but maybe we can switch over.

Okay. Okay. So the presenter mode is creating a problem, right?

We were seeing the notes. Yeah, there we, there we go.

That's it.

There we go.

Oh, thank you. Appreciate that. Well, we'll, I think we'll maybe miss out on a build, one build here, but that's just fine. Let's just go with it this way then, if this is easier for the audience. So yeah, so we were just kind of tackling the generative apps part. And in here's, you know, a number of examples. Jasper, of course, in writing. You know, you've got customer success, customer service. You've got some more of these different flavors of writing apps. I think that Adept itself is a very interesting one around actions. Early company, but doing some impressive stuff. And then, of course, we would view ChatGPT as a generative app with the foundation model underlying it, GPT-3.5, just for kind of nomenclature. And that's why, you know, you see OpenAI here as an enabler and ChatGPT as an intelligent application. And then as I referenced before on all these different enabler types, there is a, there's a lot going on in terms of not just building next-generation data-driven apps in the modern data stack, but many, many of these companies, in fact, candidly, all these companies here are helping and enabling in one way or another to build these, this generation of generative apps. One of the places that Madrona has developed, you know, kind of some of our, you know, kind of leadership in the community at convening others is something that we started a few years back called the Intelligent Application 40. These are the 40 top, and I'll tell you how we get to the top 40 companies in the private world, private companies in this whole area of intelligent and generative apps. This process starts with nominations from many of those VCs that are going to be on the, you know, kind of get on board on Sunday and that are judges and whatnot for this program. We have well over 50, you know, folks from different venture capital firms. The top venture firms make nominations and then vote on these top 40 companies. And they've really come into kind of an early— I think it's less than $30 million in capital, mid, you know, $30 to maybe 100, and then lates, you know, kind of higher than that. Enablers is a cross-cutting in terms of the capital they've raised, you know, but it's, it's that enabling layer versus actually the application layer. What's super interesting, so we had the second annual IA40 winners this year, you're looking at those 40 companies. And you'll see by the repeat winners that there were only 13 from the prior year, and they were mostly in the enabler bucket, there's a lot more persistence in that enabler layer in terms of these top 40 kind of companies. And what's also interesting to see is that, you know, a little candidly, more than half of the early stage ones are what we would consider generative apps, not just intelligent apps, either generative apps from day one, like a Copy.ai or a Jasper, or companies like Lexion that, as I'll share in just a minute, have embedded and enhanced their application, which was already an intelligent app, but they've enhanced it with generative capabilities. And that's one of the things we think is a very, very interesting trend here. So IA40, the IA40 results will— that whole process happens in the summer, gets announced in the early fall. There's many great companies here. There likely will be a good number of different ones again this year. I think it's October 11th and 12th that the summit that we'll have for that will take place. And in fact, Speaking of the summit, this is the framework that at the beginning of November, just a couple of months ago, is sort of this updated understanding of, well, what the heck is this generative app space all about? And in many respects, you know, different kinds of data types, including things like parameters and tokens, define what a lot of what's going on in the enabling layer of foundation models. And the foundation models are the lowest layer of the stack. And you've got different different types, right? You know, you've got what's, you know, OpenAI doing and then some more specific areas of what folks like DeepMind have done. You got Stable Diffusion with Stability.AI and, and, and the, some of the Stable Diffusion flexibility that they allow and, and folks have taken advantage of. And then I'll pop up over this orange section just for a second to see the different examples in generative apps. And at the time, I think we were really, you know, you know, you know, focusing, this is just less than 3 months ago, on these different categories of speech and code. You're going to hear more about Copilot. There's several other examples, CodeWhisperer by AWS and others in that code and code enhancement space. The text, there's many of them being the generative apps. One of the areas that we think is one of the most interesting areas in this generative app stack is generative middleware. I think when we were talking about this 3 months ago with folks, there was a little bit of people rolling their eyes and wondering what that really was going to mean. But we've become even more convinced now that the generative app stack is where— the generative middleware is where there's going to be a lot of opportunity for startups in addition to the generative apps themselves. So what are we learning even in the last couple of months? One of the things I think we're learning— and again, just 3 months ago at this conference, there was a lot of debate. Is there going to be just a couple models to rule them all? Is there going to be a couple foundation models to rule them all? Or is there going to be a long tail of models? I think it's become even more obvious that there will be a long tail of models. And there will be some large models that are really important and significant. But those large models will be significant in terms of the emergent properties that they enable. And then the smaller models or the long tail of models would be the ones that are contextually and domain-specific. I mean, you could imagine a model that's domain-specific to travel or domain-specific to restaurants or domain-specific to contracts, which we'll get to with a specific example. But we think that that's one of the most interesting areas of learning in the last 3 months, that it's more likely to be there's a long tail of models. Data is so critically important, not only the data that you use to originally bootstrap, but the data that you have that's differentiated data from others, and then the way you can have that continuous loop of data through effectively reinforcement learning through human feedback, which is critical and critical that you have something that is engaging if you're at the generative app layer in order to create that RLHF dynamic. Then there's this middleware topic, and I'm just listing some examples of some open source projects and some companies here that are already beginning just to form in that area. I'll talk a little bit more about that in a second. And then there's this very interesting trend of not just having, you know, de novo applications, you know, like a ChatGPT, right? But actually embedded and enhanced capabilities inside of existing applications that make those existing applications have an actionable capability around them. So how do we get that actionable capability to exist? Well, this is a little bit of a breakdown of, I think, some of the emerging subcategories of this middleware layer. Because one concept is this concept of chaining, and I'm sure others will talk about it. LangChain is kind of an early breakout success story in a number of these, but certainly in this area, just the sequencing of large language model calls. And then the ability to do prompting and prompt management and prompt optimization across these different kinds. It could be more than one large language model. It could be multiple times within one large language model, which then leads to this whole next area of, well, okay, I'm relying on a certain underlying large language model, maybe a second large language model, or maybe something that's more customized and more contextually specific. But I also can have knowledge resources, knowledge sources that I want to pull in in terms of being able to deliver the kind of thing that my application is looking for. And then that takes us into this next level of actioning. And in actioning, I think you're, you're really talking about this ability to, you know, it's one thing for, you know, you know, some kind of a generative app, go back to my travel example, to produce an itinerary for me, but I really wanna be able to book the trip. And so how do I do that? And is there an agent that could be created for booking the trip as it were? Or deploying the code or whatever might be the action that you want to have on top of the generative output. It's very important to be able to persist. If you're going to have multiple calls, you're going to have multiple prompts, you're going to have multiple models underlying that. What kind of a persistent memory layer exists? And then what happens when GPT-4 comes along? Is it going to break your generative app? Are you going to want to be able to test it first and see whether or not GPT-4 breaks it in some way? And if so, you've got to do some modification before you deploy that version. And so the ability to evaluate and assess— there's also issues related to model drift and whatnot. So quite a lot going on here in the middleware layer. And now let me just touch on this one last concept, which is this idea that there's going to be new and enhanced apps that can both succeed. And so, you know, the new ones I think are kind of self-evident, right? You know, on the consumer side, we've got a lot of— and I know that some of these things get used for commercial purposes as well. But, you know, I mean, the numbers are just astonishing, right? On, you know, the adoption of ChatGPT by, I mean, some reports as many as 500 million people have played with it at least once, and there's over 10 million, you know, queries per day already, which is just a phenomenon. But you've got lots of other examples, Midjourney, Lensa, which has become an exciting de novo app. And then you've got these de novo apps. You're going to hear from Chris from RunwayML, Jasper.ai. You're going to hear about the story of Copilot. Those are all new apps that are more professional-oriented in their focus. On the consumer side, again, you can come back to these enhanced apps. And you've got great productivity tools, whether it's Notion or Coda or Smartsheet or others that are embedding or enhancing their existing application by embedding, you know, capabilities that they've built on leveraging foundation models. And sometimes it's just foundation models specifically, sometimes it's a foundation model or even some kind of a finished ChatGPT-style application. And, you know, you know, and then you've also got that on the professional side or the commercial side. You know, one of the examples that, I've referenced here, so I'll just articulate it a little bit more is Lexion is a company that was already an intelligent application business. They did entity extraction in contracts and they could help you manage your contracts, anticipate contract lifecycles for business contracts. Of course, they built up their own differentiated dataset by doing that. They were able to take GPT-3 and enhance it with some of their own knowledge base. And now they're generating clauses, contract clauses. So they've developed a contract clause generation capability on top of their contract management capability. Tesorio is doing things around cash management, and Figma and others are embedding these kinds of enhanced generative capabilities. So a lot of things to think about there. I think the focus of this week is going to be generally more on some of the new de novo apps, but to the extent that you're going back to your day job, as it were, and thinking about these different ways to leverage foundation models, some of the enhancements to existing apps are going to be quite powerful as well. Then that leads us to a bunch of questions then, and I'm going to set up these questions, but then really open it up to the audience for other questions that they have here. Maybe Mike or somebody on the team can help facilitate that with me. But let's just unpack a couple of these questions for a second because I think the first one is actually the, the one that all of us that have aspirations to, to start and to build companies, you know, have to be thinking a lot about. You know, there are clearly a lot of big players in this space, but you know, what's unclear, you know, as evidenced even by the difference in revenue generation from Jasper.ai relative to OpenAI last year, right, is where and how will the value be captured? Not just created, there's going to be tons of value created here, but where and how is the value going to be captured in this stack? Is it going to be at the foundation model layers? Is it going to be at the big foundation models versus some of the contextual specific foundation models? What about that middleware area that I started to unpack a little bit in this presentation? There's history of some big successes in middleware of de novo startups. You can kind of look back at, you know, do some respect things like Zapier for an example. And, you know, think about, you know, kind of MuleSoft and you can go way, way back and there's other kinds of examples of application servers in the middleware space. Generative apps themselves, which generative apps are gonna be able to be more persistent value add? You know, will a Lensa be able to persist as a, you know, kind of a service that people use and pay for over and over again? Or is it gonna be the next generation, you know, travel agent that is actually a highly automated, if not fully automated travel agent. We talked about the middleware a good bit and what that's going to potentially— some of the pieces of what that's going to be. The other important piece here is the interplay between the bootstrapped data and my ability to have great engagement so that I can generate reinforcement learning through human feedback. And be able to create my own data flywheel in my generative app? And how do I and who do I partner with in terms of doing that, which may not be the underlying foundation model that helped me bootstrap it? Another question here is that where are the new apps and enhanced apps going to prevail in terms of being winners? Is the, for instance, the industry and functional areas, is that going to be more advantage to enhancing existing apps whereas there could be other slices that are more of a de novo app. And then I think that as we're already starting to hear a little bit about what are the roles of apps that are going to exist in the cloud so they can be truly multiplayer, easy-to-use cloud apps. And then what about at the edge as we're already starting to hear little bits and pieces, maybe Zach will tell us a little bit more about it on some of the things that they're already experimenting with back on Apple devices in terms of ChatGPT. App at the edge type of thing. And then this is a little bit more of a sort of a, maybe a more fun question to speculate on. Clearly Microsoft and OpenAI have been working very closely together to some extent, AWS and Stable Diffusion. Google's got incredible capabilities in-house between the Lambda work, historically the BERT work, of course what DeepMind's doing. But how will this disruption, if it is in fact as big as I believe it's gonna be, play out in terms of the winners and losers for the very biggest companies, not to mention all of you that are going to try to create that disruption by building startups. And so I have this last slide. This is a real slide, by the way. This is a real picture from Halloween down in Silicon Valley just a couple of months ago. And as you can see, virtual reality, it's speculated here that they've kind of met their life and their death. I don't know if that's fully true. You could have maybe added another Tombstone for crypto. Again, not necessarily fully true. But what about artificial intelligence? And interestingly enough, you know, they had picked kind of 2012. That was right about when we started investing in this thesis. And, you know, it's— I'm a believer. I think that this is— there's going to be hype, there's going to be overhype, but there's a big amount of substance here in connecting the dots through the capabilities that foundation models, middleware, and generative apps are going to produce. I'm so excited to see what comes out of the the competition this weekend. And I think we got just a couple of minutes here for any additional questions.

All right. Thank you, Matt. That was the perfect setup. And really excited to jump into some of these questions. So, and you won't be surprised. I mean, this is the 70+ participants on the 18 teams. You know, they're looking for guidance on where to prioritize, where to focus their time this week as they build. So one question came from Himanshu. Do you think the middleware will create custom models for verticals, healthcare, retail, et cetera?

Well, I think that's a great question. I, I think that the middleware, I would just, I would think of it as more of an enabling layer. And so let's take in, you know, in your, in your healthcare reference there, you know, and so, you know, a foundation model, you know, with, with its emergent properties is gonna be helpful, but you're probably gonna need a combination of, some domain-specific models, and some knowledge resources. And you're going to have to bring that all together to present it into an app. Maybe it's an app that tells me whether or not I've got the flu, right? You know, based on some visual input that I give it, you know, a picture, you know, or some other visual input. And so the middleware layer, I think, becomes this enabler so that I can have, you know, multiple interactions, multiple iterations. I can retain state. I can bring in these, you know, different resources. And in this case, you know, wouldn't you love to have an action or, you know, built in through some agent that actually, you know, you know, kind of orders me from, you know, you know, PillPack that the, you know, the medicine I need based on what the result is of my generative app, you know, kind of generating a diagnosis. So again, I'm being fanciful, but I'm trying to make the point. It's the middleware I think is going to be enabling the generative app to, to, to be created. And I think it's gonna be a necessary layer in order to get, I, I don't think the generative apps are gonna wanna produce all this stuff that's in the middle that is gonna be persistently needed across what I believe is gonna be literally thousands and thousands of different types of, in, in, in domain-specific, you know, vertical-specific generative apps being built. Great.

Sticking with middleware, we have a question from Jeremy around market timing. That as you think about building the middleware, there's really that the target market is nascent, it's just coming online more and more. How do you think about getting comfortable with it? There's a big enough TAM to address with some of these emerging companies?

Well, I think that's a great question about, you know, how early it is. You know, these, you know, foundation models, while they've been, you know, really being worked on for about 5 years. The ability to connect the dots and sort of unlock the power of them is really, I think, kind of, I think we'll look back and say it comes from the summer of 2022 when you started to see things like Midjourney, DALL·E 2, Runway ML in its earliest forms, Stable Diffusion kind of come onto the scenes. And then obviously a few months later, ChatGPT. And so with that, it's even earlier still because those had to be built in a sense themselves by some very talented, some very smart teams. But they're starting to notice this trend of things around prompting and prompt management. They're around chaining together. And what has happened, I think, quite organically in the open source community with with offerings like DUST and then LangChain is, I think, quite impressive. I think LangChain launched just around the time of 3 months ago of this Intelligent Application Summit that we were having. I think Harrison, who I think is speaking as well during the course of the week here, was actually at the Intelligent Application Summit. And since then, I mean, it's already got over 4,000 stars. It's got incredible engagement in its Discord. And so I think that you're seeing, while very nascent, this middleware area emerging. I think you're right, it doesn't tend to be the very biggest by category sector. I think there will be lots of capital that goes into and some big bets on the foundation model side. And then there'll be just thousands and thousands of generative apps, but there'll be some set of, and I don't think it'll just be one, middleware players that listen to the customer, listen to the market, And I think candidly focused more on creating value initially, and then over time figuring out what's a logical friction, low friction way for them to capture value. That'll create, you know, a handful of really big winners in that middleware space.

Great. We had another question here about seeing more value in enterprise versus consumer apps. Now I know that, you know, Madrona has in Seattle in general quite a history in enterprise software, the cloud, ML, AI. But then again, you've mentioned travel as a scenario. There might be an opportunity to reimagine key consumer verticals as well. How do you think about those two and trade-offs there for these teams?

I think there's sort of this horizontal layer of things, like what we've seen with a lot of the different text-generating applications, going back again to the Copy.ai and other sides, and of course, Jaspers. Now ChatGPT. So there's horizontal layers like that. I think there's going to be a lot of either functional, sales-specific, marketing-specific, customer service-specific applications that are enterprise. And there, I'm not as clear if it's going to be the de novo app or it's going to be somebody who's already built an intelligent app. I mean, there's lots of intelligent apps out there. There are already teams that are pretty savvy in their machine learning and deep learning skills. They know how to deploy models in production that are taking something that otherwise might be called software and actually turning it into dataware. So I think that the enterprise side, it's a little harder to tell. I actually think, and people would debate this with me because they'd say, hey, they don't have the data. But I think if you can have creative ways to bootstrap the data on the consumer side, that designing from ground one, a next generation travel service, that's consumer facing, it's engaging, it's simple, it's intuitive. I mean, And this is something I didn't mention as much before on the consumer side. We all want to have something that's more personalized and less friction. So wouldn't you just love to have a travel itinerary that's personalized to you? I mean, I'm a Delta frequent flyer, so, you know, and an Alaska frequent flyer. I'm not a United or American frequent flyer, right? So I want to get booked on those airlines. You might be something else if you live in San Francisco or you live in Dallas. So that kind of personalization, I think there's some opportunities for some really clever teams to think deeply about how to unlock the power of this technology and solve the consumer's problem better than it's ever been solved before. So time will tell. You know, you got to, you got to be able to have access to data. You got to be able to create those data flywheels. But I think it's going to be, it's going to be an interesting, interesting race.

Outstanding. That's a perfect kickoff to our speaker series, really sets the foundation for the rest of the conversation we'll be having. and thank you so much for joining us at the event here.

Well, thanks very much, Mike. My pleasure. Can't wait to see the results.

Awesome. Okay. Thank you, Matt. And we are getting prepared here for, for our next speaker. We'll pull up a slide here in a second.