Kleiner Perkins at Cerebral Valley: Exploring AI’s Frontiers with Leigh Marie Braswell & Bucky Moore

I am very excited about this panel. I feel like we're going to get to lay out some of the big themes sort of floating the surface of the conference. We've got some VCs. They can argue with each other a little bit. Uh, we've got Bucky Moore on the left, partner at Kleiner Perkins, invested in Tecton, Teleport, Materialise. Leigh Marie Braswell at Founders Fund, previously at Scale. And Amber Yang, investor at Bloomberg Beta. And we're gonna start off with you. You're to sort of blame for Cerebral Valley. What— explain sort of how Cerebral Valley like came into being, what you think about the term, like, yeah, what's your take on it?

Yeah, so I first heard about Cerebral Valley from one of my very good friends who was working at Lyft and during the pandemic moved to New York because he wanted to, you know, have fun and then was unfortunately laid off during like the big tech layoffs and then was sort of having an existential crisis and was like, you know, I don't love SF, but I really want to be a part of this big movement of like AI excitement, which is really revitalizing the city. And he claims he heard it from some other friend who called it Cerebral Valley. And he's like, I really wanna be a part of Cerebral Valley, and that's why I'm moving back to SF. And so when I heard this, I was like, oh, I guess this is a thing, which led to me tweeting about it. But overall—

And the tweet just went wild, and then SF Standard wrote it up, right? And now we're off to the races.

Yeah, exactly. And here we are today.

The power of Twitter. So feel free to steal quotes and throw them up and pull us out of context. I mean, I want to start with the paper. I mean, the paper, I think, is on people's minds, this sort of, like, pause for 6 months. I polled both readers and some of the people here. Like, obviously, this is not a crowd of people who thinks we should pause for 6 months. But I don't know, what— does any of you think we should, or you're all against?

I'm probably against.

Yeah, you're all against.

Can you hear me now?

Yeah.

Yeah, I'm against. I mean, I think it's really— I think, of course, everyone believes AI safety is important, and we're already seeing a lot of the risks that this technology can introduce, like in real time around synthetic video generation, synthetic image generation, voice generation. A lot of really bad things can happen with those technologies. And so, of course, safety is important. But if you think about who we're competing with on a global basis and pausing for 6 months when they're not, to me, the cost of that is exorbitantly high.

Yeah, I mean, it can feel almost like promotion for the AI companies, the idea that we need to pause, that they're so close. I mean, do you worry that we are, or do you think we are close to a form of technology that will get out of our control? Or, I mean, people talk about sort of a self-programming, you know, AI. Like, do you think we're close to the point where any of the AI systems are self-improving?

Yeah, well, I hate the first thing that I'm going to say, so being such a dark thing, but I mean, I think we're already at the point, even with open-source technology, so even if we did do the pause and we just kind of let things play out, where some really dangerous stuff can be done, right? People can be impersonated, news can be faked, both with images, videos, dangerous forms of content can appear online. So, I mean, I do think more broadly, we do need to think about ways to regulate this, but with the technology that we already have. I wouldn't say that with the self-improvement, we're really there yet. I mean, I think right now, in terms of CodeGen, we're at the point where most developers can be aided by codegen technologies, especially with the rise of, you know, now Copilot for on-prem and stuff like that. But yeah, I was— I think my, like, this is really happening moment was last year around, like, middle of the year when most of my developer friends were using Copilot. I don't think we're at the point where we can take them out of the loop.

Yeah.

And that's the theme that's gonna be coming up. You know, do coders get cut out? I'm talking to Replit at the end of the day. That's certainly on my mind there. I don't know.

Yeah.

What do you take away from, like, the letter and some of the fears? I mean, we're clearly in this arms race now with AI, or do you have a point of view on it?

Yeah, I think it's interesting where the letter, it's unclear to tell, like, where people's motivations are coming from. And it's clear right now that, like, the power of foundational models lies, like, in a few hands. And the letter almost promotes this idea of, like, okay, if these few big hands pause, then this will lead to more of a fragmentation in the market where people, other more open-source developers start building. I'm not entirely sure if that's a good thing either.

Right.

It'd be great for the open-source world, obviously, if OpenAI just said, okay, we're going to stop for 6 months. You're all looking at startups, making investments. I'm curious on this debate between the company needs a great research arm, foundation model, some of the top engineers, Or, you know, it's a product company, it knows what its users want. Like, obviously you'd want all of the above, but do you have a general leaning either way in terms of what bets you're willing to make? Would you invest in an AI company that says, yeah, we're just building on great tools that exist with a product understanding of what our users want?

Yeah, so my framework for thinking about this is there's sort of 2 types of startups today. Some are founded by AI practitioners, and therefore that research arm is in place from the get-go. The other are software companies, people who are proficient at software development and not necessarily AI experts by trade. I think what you're starting to see is this pattern where that latter group starts off by riding on the shoulders of giants, aka these APIs that OpenAI is giving us. If it becomes pertinent to start to differentiate on the quality of the model and they want to leverage the data that they're capturing from their customers and keep that proprietary, you're starting to see this pattern where they are actually starting to invest in GPU infrastructure and hiring the requisite people to actually work with that infrastructure and train these models. Jasper's a great example of this that I think has gone down that path after starting in kind of a more— I hate to use the term wrapper, we can talk about that too. But to me, I think both companies can really— both form factors of companies can really work. But I do think that at some point, these companies that are riding on top of OpenAI, the question they have to ask themselves is like, to what extent do I want this to be a competitive advantage? And if you look at what the incumbents are doing, say take like a company in our portfolio like Coda that brings GPT-4 into their product for the benefit of their users, You could argue that that's a new feature that they're building, but by no means the differentiating feature. And I think in that case, it makes a lot of sense to leverage that. But when it does need to be a competitive advantage, I think you're gonna start to see, as this expertise becomes more democratized, as the cost of compute goes down, you're gonna see a lot more companies that are running their own infrastructure.

Any disagreement with that, or no? I wanted to ask Leigh Marie a question. I mean, we just heard from Quora. company that's sort of like an older startup pivoting in. You worked at Scale. Like, I mean, there— and there's like the Notions, I think Canva announced something. There's just lots of sort of what I would think of like older unicorns who absolutely want to grab the generative AI trend and, and, you know, claim it for themselves. Like, what does that mean for new companies? Do you think— I mean, it's hard to say for each of these companies. I'm sure it's unique, but how do you read the landscape in terms of the incumbent unicorns Sort of fending off upstarts in this space?

I think that's a great question. And I mean, I think we have to sometimes zoom out from all the hype and just being so LLM-specific. And yeah, think about the incumbents in the room. So I mean, I guess I can talk a little bit about Scale. Scale's Act 1 was providing labeling data, both image and LiDAR data for autonomous vehicle companies primarily. And by doing that, they basically created an operational backend where they can sort of manage tens or hundreds of thousands of people around the world. Automatically. And that same sort of technology becomes very helpful with sort of the large language model paradigm because you're seeing, you know, ChatGPT was released with something called RLHF, reinforcement learning with human feedback. That requires a lot of human labelers. And then also the last panel was talking about model eval. The way that a lot of these models are evaluated, because it's subjective and it's text, is through humans. And it needs to be done very often because what's relevant to us one day might not be very relevant to us the next day, or I might want my model to do something differently. So I think Scale in particular is a really interesting example of like an infrastructure company that's already built something really valuable that's now kind of translating over to these new areas.

Turns around humans who just sit around reinforcing these models over time, making sure that they're—

We still have a job and we're alive, you know, like—

Our job is to make sure the models are smart. Yeah.

Yeah, exactly.

Exactly.

In terms of like investment, especially at the early stage, I mean, sort of touching on the question I asked Bucky, Where it's just, what types of companies are you looking at? Just more like categories, or how are you hunting for startups at the moment?

Yeah, I mean, I think there's— right now I'm seeing a lot of applications that are basically like these wrapper-type companies around APIs. And then there's also a new crop of ML infrastructure layer things that I've been seeing. I think there's a lot of negativity around this idea of these API wrappers, but I think I have a bit more of a positive take. I think there's a lot of noise right now, but if you look at founders starting ML companies 5 years ago, they were all pretty much researchers, people who had PhDs in AI. And I think the ability to be able to build something super quickly right now with AI just democratizes access for different types of experiences. And that also leads the people who are more product-focused and product thinkers to be able to build really sophisticated types of applications. I think in the long term, 5, 10 years from now, they won't necessarily position themselves as AI-first wrapper companies, but more so as just, this is a really good product and it's backed by AI. That fact that it's an AI wrapper becomes a little bit less relevant.

In terms of the foundation model companies, I polled some of you, and certainly OpenAI is seen as in the lead. I think Anthropic seemed like I think we've heard that from some of the panels today. I don't know, what's your view of how well positioned— do you think we're going to have multiple successful foundation model companies? Are you worried about the dominance of OpenAI? How do you think about that, especially in terms of startups? If OpenAI is so far ahead that anywhere they build is a threat to your company?

I think OpenAI has a number of things going for it right now. They've got the partnership with Microsoft from both a capital and infrastructure standpoint. I think being the first company to have a truly kind of like consumer-centric mandate of creating value with AI for consumers, that attracted a lot of the best people. And I think you see that in the team that they've built and the rate of progress that ensues. But at the same time, they've, they've had to make some tradeoffs to get to the point that they're at today. And I think the Microsoft partnership is a really clear example of that, where there's a huge universe of companies for whatever reason, either because they consider themselves competitive with some part of Microsoft Or they don't necessarily want to be as tethered to an entity that owns GitHub as well. What you're starting to see is that a company like Anthropic is just seeing an incredible amount of interest on that basis alone. I think the other thing I would say is that the rush of companies right now that is pouring into trying to actually onboard to these APIs, it's more than most of these companies can really handle at this point. There's just so much interest to go around that I think it's pretty easy to see how a lot of these companies get off to a hot start. OpenAI, of course, already has. But I think in the long term, what we're going to start to see is that the surface area of how all this technology can be applied is just so impossibly broad for one company to cover that I think it's necessary that you're going to see many of these companies rise up. How many, I think, is a very open question. And as things change so fast, it's pretty difficult to take hard stances on hypotheticals like that. But what I'm certain of is that the surface area is already impossibly broad. And it reminds me a little bit of in the early days of AWS when I think everybody was sort of just envisioning a future world where you would just use AWS APIs for every bit of your infrastructure. And what you're seeing today is like very few people are actually starting from that point. And the reason for that is because again, AWS just hasn't been able to keep up with all the form factors of infrastructure that developers want to consume. And so I sort of use that framework and I think about the future playing out very similarly.

Yeah, definitely feels like there's a lot of fear getting too locked in with Microsoft. especially if you compete with Microsoft. That seems, yeah, I feel like I'm hearing that a lot. I don't know what views on the other foundation models, or, I mean, there are a lot, you know, we're gonna be talking to sort of Adept, which has its own sort of particular focus. And then I, maybe I'll throw it out this way, like the open source in particular, you've got sort of, do you think the open source sort of amalgam of things is a competitive answer to OpenAI? OpenAI and Anthropic? Like, do you think people are gonna be able to pull quickly from whatever ChatGPT-4 is and just implement that in open source? Like, what's your view of the open source suite of offerings?

I think it'll be really interesting to see what happens with GPT-4. I mean, I think in terms of use cases that you can solve using GPT-3, you can already do that with open source now, and you can train like an equivalent model. You're using like— Mosaic wrote a great blog post, Like $500K, you can get a GPT-3 equivalent model. That's really not like this capital moat that we're used to seeing the foundation model companies talk about. I'm very curious to see when somebody's going to come out with GPT-4. Is it going to be a year? Is it going to be longer or shorter? I think that'll be a good case study, like how much is this accumulating advantage actually happening within OpenAI versus the open source and some of the other players?

I would just add on this that I heard Adam earlier say that he's starting to realize there's a lot of use cases for which these pretrained open-source models are good enough today. I think that that's only going to continue, meaning more and more of the use cases that people want to pursue will be— their needs will be met by these open-source models. I think as we start to hit an asymptote in terms of more data, more parameters, more compute, not necessarily unlocking meaningful more performance and utility, I think we're on a crash course towards open-source pretrained models being everywhere. And I think the end state is somebody's going to release one with the weights, and that's going to force the entire industry to rethink whether that's something that they have to do to be competitive as well.

Sorry, the weights?

Yeah. So today, if you have a model that's pre-trained and off the shelf, what you can't actually get into is the weights. So you can use the model, you can fine-tune it, but you can't actually look at how they've programmed those weights. I believe that's going to change at some point because of just competitive forces, right? Anything that's this useful tends to trend towards commoditization, and this is very useful technology.

Amber, have you tried to like build, like use ChatGPT for investing? Or I'm curious, I'm like starting to think about how as a reporter I could, you know, train it on my writing or whatever. I ran into like all sorts of memory problems with ChatGPT, but I'm just curious in terms of like a tool that you actually find useful, have you found ways to implement it in terms of looking for startups?

Yeah, I mean, I think ChatGPT is definitely a part of my workflow today. I think everyone, every knowledge worker will have that moment at some point where you're like genuinely concerned that your job can be taken over by ChatGPT. Um, and I think I've hit that point a few times.

In terms of it like identifies a startup that you would go after, or—

Like, for example, we recently did our like quarterly review for our fund, and I used ChatGPT to basically classify like a list of all of the companies that were funded this quarter and whether or not they were in scope for us. And this is a job that previously we had done manually and would take hours, and It was done within like 5 minutes.

Crazy.

Yeah. I think it's really interesting, right now the way that you interact with ChatGPT is in this chat interface, but I think a new world of envisioning the way you interact with models will be a workflow of various types of prompts that you prompt it with, and it'll act asynchronously as a bot that can do research for you latently. Those are all extremely promising to me as someone who spends most of my day doing research and diligence.

I wanted to transition to just the money of it all and valuations. I mean, we're in this crazy world where it feels like an AI startup can raise a $4 billion valuation, whatever OpenAI is, $29 or something. Meanwhile, poor software companies, we're used to these great multiples and now they're getting Destroyed. I wanted to put out this idea, and I'm sure this— I feel like this is seeping out now, just the idea that, you know, there's a little round-tripping going on, and that some of the big tech companies, you know, know that the money they're investing is gonna come back as revenue to them if they make these investments. And I'm curious, just as investors, like, how do you compete with that if the sort of incumbent tech giants have a reason to invest above the price, yeah, are you still relevant or are these companies just going to take all the great investments?

Yeah. The first thing I'd say is, as a firm at Kleiner Perkins, we're extremely high conviction and bullish that there's going to be some enormous companies that emerge from this space. We're certainly acting accordingly. We're trying to find every single one of those and make sure we invest in the right ones. At the same time, to your point, the way I would think about that is these are really capital-intensive businesses. businesses. The reason they're capital intensive is because of the talent, the compute, and the infrastructure spend that goes into training and developing these models. When you combine capital intensity with a universe of what I'll call non-financially motivated investors, well, maybe non-financially motivated is a loose term because of course there's a revenue opportunity for these people, but not necessarily seeking to generate a return in the venture sense. You have a different cost of capital available to these companies. When they're very capital intensive businesses and there's a different cost of capital being waved in front of them, it's very, very compelling to those businesses to want to seek that out and take advantage of that. And so I think the challenge for a lot of venture investors is just they're underwriting to a very different sort of return than these financially motivated— I'll keep saying that— strategic investors are. And so I think that's why you're seeing a lot of these large rounds be led by strategics, and that's why you're seeing, I would say, fewer venture investors and even late-stage growth investors be as active in these foundation model rounds.

Uh, I mean, Founders Fund is reported to have invested in OpenAI. Can you say anything about that?

I can't, unfortunately.

Uh, but, you know, that's out there. I mean, do you have—

Uh, I tried.

I mean, yeah, I mean, what do you think? Can you— what's your— I mean, the, the valuation just seems like super high for OpenAI, or like, what's your assessment of Just like whether that's ahead of the fundamentals.

Yeah, I mean, as a venture investor, you kind of always have to zoom out, and it's very easy, and I do it all the time, to sort of like write something off. Oh, there's this problem, there's this thing, maybe it won't work because of this, maybe it won't work because of this. And I think with OpenAI, it's beneficial to just take a step back, and if all we're saying is true, like this is theoretically on the infrastructure side, a new cloud tax. It's like a new percentage of like a software company's revenue. It will go to the cloud, like for cloud compute, as we've been seeing. And then some will probably go to a large language model provider, whether that's a foundation model or an open source. And so if you take that view, that could be hundreds of billions of dollars. Plus you've got OpenAI's application development. Maybe they go into search and take some of Google's market share. One percentage point, that's $2 billion. So I do think you have to like sort of take take a step back, I do see how like sort of the— that's right.

Again, polled people, and I think it's a— it was about 50/50 in terms of people who think OpenAI will be worth $500 billion in 10 years. So people, people are on both sides of that. Um, I'm curious in terms of early stage, like there's this dynamic now that it's like hard to get the compute power, or I'm like, I was talking to someone yesterday and I was like, isn't it a market? Like, shouldn't you be able to just like pay what you want and that changes the price. But it seems like people have like already pre-negotiated like Google Compute and stuff like that. So then there's a feeling of scarcity. Like, do you worry with like the startups, the early stage, that they're having trouble just like getting access to the GPUs that they would need to build their businesses?

I mean, I think it's an interesting like problem that I feel like will be a bit more felt later on. I think right now there's just such a wealth of people, of resources that are being handed out. I feel like every VC right now and every provider is just trying to get in touch with as many builders as possible.

Lock them in as customers and then let the market set the price.

My sense is the supply shock around GPU compute is rooted in, one, the supply chain chaos from the pandemic, two, the confluence of Bitcoin mining and AI model training surging at once. And I think now that the former is, is probably a little bit less profitable, you're going to see the demand curve sort of change accordingly. And I'm already starting to see that, like, if you look out a year from now and you talk to, you know, some of the obvious suspects who are producers and large consumers of, of GPUs, that I think there's, there's better times ahead and there's going to be a lot more compute available.

Right. I mean, I'm going to butcher this, but Nvidia basically said something to the extent, oh, we don't really like the crypto thing. We're very happy to see it sort of shifting to AI, which was fascinating. You're seeing some of the crypto mining compute use case shift over to AI, freeing it up. So that's super interesting.

What's—

I mean, going back to this whole like Cerebral Valley thing, just because I think it's so funny, like what is the vibe in San Francisco right now? Like I feel like we've all like— I'm not in touch with the youth anymore, I guess. Like there was like a crypto— like the crypto community, I got it. Like I don't know, how would you paint a picture of what the AI community is like right now? What's it like at Stanford right now with people working on this? What's your read on the vibes?

Yeah, I think the vibes are great. I think that there's a lot of energy that's actually happening in San Francisco rather than in South Bay. I think if you wanted to, as a developer, you could be going to multiple events every single night. There's hackathons every single weekend that are being sponsored by VCs and also people just clamoring to get you to use their product. And yeah, I think you start running into the same faces at every single event. And I've known many friends who found their co-founders by just running into people 3, 4 times a week.

I'm gonna open it up to questions if anybody has one. I'll ask one more before that. Yeah, I mean, do you think Microsoft's ahead, or like, what is your view in terms— any hot takes? I mean, people are very curious in terms of like Apple, for example. Like, any hot takes from any of you in terms of where big tech is positioned in this sort of AI arms race? Open to whoever.

I don't think it's a hot take, but if you just take a step back and look at what Microsoft does well, it's enterprise and it's distribution and enterprise. We've seen companies like Slack and, you know, perhaps more recently Notion sort of bump up against the reality of that, right? And now that every single Microsoft product is going to have the most cutting-edge language models embedded inside of them, I think those products are going to become even more differentiated. So if I'm a CIO at a big bank, I'm not just buying these products now because, like, I like doing business with Microsoft and having so-called one throat to choke. I think I'm going to start to shift my mindset towards, like, I'm buying these products for that convenient reason, but also because they have the best technology. And if I was competing with Microsoft in that sense, that would scare me a little bit.

Any questions? Sure, go ahead.

Thanks to the panelists for the amazing insights. I'm sorry, I'm going to talk about the letter again, if you don't mind. So going back to the letter that everybody's talking about these days, the intention for the letter was just to get us to talk about it. I don't think they literally meant we should stop research for 6 months even though they had that, but they knew that would trigger us to have a conversation. It goes back— I'm one of the signers, by the way, on the letter. I don't agree with 6-month pause, but I agree with the spirit, which is this is really scary. If I build a model of Bucky and I run an experiment like AlphaGo where I play against Bucky 1,000 times, 1 million times, 10 million times to convince him with a pitch, I will come up with a perfect pitch that he will say yes to and get fooled into saying yes. And that can be used for that, that can be used for controlling citizens, that can be used for controlling populations. It can create damage at levels that we have never seen before. So the question is, are you concerned about that or not?

I'm very concerned about that, both personally in this example, but also globally. And, uh, and Amr, what I would say—

I know, now there's a gauntlet to hack your brain here.

I think it was Leigh Marie and I who reacted to that question, and sort of speaking for both of us, like, I think we actually are in a similar position to you, which is like the pause for 6 months seems destructive and probably exorbitantly expensive to consider. But at the same time, we have to have this conversation, and there's a lot of really serious problems already in front of us and more to come that we have to solve. And so the thing that I would say is I worry that the outcome of this conversation will be some kind of like centralized body that is presiding over what's right and what's wrong, which historically just hasn't worked really well for society in any construct. And so I hope we can come up with something better. I hope we can come up with something that's clear, that's objective, and ideally as decentralized as possible to have that conversation. And I agree that like the positive impact of the letter is that we're having it, the conversation that is.

All right, that was a great question. So I think we'll wrap up there. We're about to go to lunch. Come back. We're gonna start sharply at 1:30. You've already picked your lunch option. Network, join some group threads, you know, get an investment. And thanks, thanks so much. Looking forward to seeing you in the afternoon. Thank you.