Workshop Wednesday: For b2b Generative AI Apps, is Less More? with Andreessen Horowitz

Welcome, everyone. So many of you so quickly today. Welcome. All right, thanks for joining another workshop Wednesday where every week we're here hopefully to have an impactful discussion. I know this one will be just the same. So thank you so much for joining. We're just going to give it a moment as people are coming in the zoom this morning, this afternoon, this evening, wherever you're tuning in from, if you do want a message in the chat stating where you are tuning in from, we love to hear. It's always super exciting to see people in Asia, people in Europe, so definitely drop it in so we can see where this community is tuning in from. All right, so we have Canada, Mexico City, sunny Florida. Love that ad. San Jose, Colorado. Love seeing it. Thank you so much, guys. Okay with that. We're gonna start kicking things off. Thank you so much. We have Christina Shen, general partner at a16z. She is going to be discussing for B2B generative AI apps is less more. I know gen AI is a super hot topic right now. So really timely session and we're so excited to get your questions answered today. As a reminder, when you're putting questions in the chat, I'll be messaging you individually. So make sure you keep an eye on that meeting chat. I'll be asking if you want to ask that question out loud at the end of the session. Of course, we encourage cameras on and a lot of interactivity. It just makes the session better and gets your questions answered in a more effective way. But with that. Welcome, Christina. Thanks so much for joining us.

Thanks so much, Caitlin. I'm so excited to be here. It was awesome. Yeah.

Well, to get to know you a little bit better, we're going to dive into a few questions if you're ready.

Let's go.

Always ready. Love it. What is one of your favorite investments? You are the firm made in the past year.

Oh. So it's always tough when you ask me, you know, who's your favorite baby? Have multiple babies at home. So portfolio companies are hard too. But you know, I probably have a soft spot for we invested or I led our investment in a company called Equals. So Equals on app. They are a next generation spreadsheet that is AI enabled and the reason why I'm so excited about them is, well one, I used to be a nerdy financial analyst that spent half my time in Excel and you know, I probably pride myself on a crazy number of shortcuts and being faster than everybody. And Bobby Pinero, who's the founder, was kind of the first business hire at Intercom. His co founder, Ben was the first engineer at Intercom. Both spent, I want to say, six to eight years at Intercom building out the business. And they jumped out to start this company because they wanted to bring the power of AI to a native spreadsheet as well as the power of the modern data stack. So think being able to connect to your snowflake, your databricks, any of your SaaS apps to live connect data and then get the supercharged power of AI to actually make you way more efficient and effective in running analyses. And so I always love when people are building products that I want to use on a daily basis. And so that's probably soft spot for me since I've known the team for over a decade and it's exactly the product that I want. So we invested in them a little less than a year ago. And you guys should check out the product.

Yeah.

Wow.

And an Excel wizard, I wish. So congrats on that as well. Okay, what has most surprised you about venture capital this year?

Gosh. Well, so I've been in venture capital now for 12 years, I believe, so it's been a crazy ride. I watched the whole bull market starting all the way back in like 2010, 2011, back when, you know, you know, 10 times revenue was insanely expensive for a SaaS company. And you know, kind of watch the SaaS and Cloud Wave ride as I was investor at Bessemer and then now at Andreessen Horowitz leading our SaaS practice. And so, you know, what actually surprised me is I think, I think thought there was going to be a huge pullback in the markets this year. And you know, obviously we're down from 2021. We can't compare to 2021. But I have seen a tremendous amount of innovation in the last year. I think a lot of it is driven by AI and generative AI, which we're going to talk a lot about today. And I think we're in such the early innings of what is possible. But I forgot who tweeted it. I think it was Gokul. I've seen more kids drop out of school to start the next company because they can't wait. They can't miss this wave of LLMs and what is possible with companies they can build. And so I think that's what I'm so excited about is it's a really unique time. There's never been a better time to build a company. There's a lot of capital in the markets, there's a lot of innovation we're in a platform shift with AI and I think, you know, who knows what's to come? But I actually am quite excited for. I know we're near the tail end of the year, but the rest of the year and really excited for 2024.

Yeah, I'm sure you'll have a lot of questions about predictions for 2024 as well. Who is one of your favorite co investors? I know we're asking you to pick a lot of favorites today.

I know. I'm like, scared someone can join this and be like, christina, how did you not pick me? You know, I think I've always. So I was at Bessemer for eight years of my career, and I started very young and worked my way up. And it's very hard not to have a soft spot for your mentor. So I have to give an easy call out to Byron Dieter. You know, he hired me when I was very young. I somehow tricked him into it. You know, I learned a lot from him. And we have some co investments now, so one of them is pave in the compensation and data space. And hopefully we'll have more together. But it's hard not to have a soft spot for someone who kind of raised you.

Yeah, what a sweet story, too. With that, we will let you start the presentation because I know a bunch of questions are already coming in, so people are excited and ready to roll.

Awesome. Well, let's get right on in. Well, I'm very excited about this topic because I spent a tremendous amount of time thinking about how is AI going to impact the software and SaaS landscape? And not only what does the next couple years have coming for us, but what does it look like in five to 10 years? And so everyone talks about gender of AI, and actually in the B2B world, I would actually poise the question of in the next couple of years is actually less more. And what I mean by that is, you know, we talk about gen AI, right? Generative AI. And what does that mean? That means we now have this incredible power thanks to OpenAI and a bunch of foundation models where you can start with a text prompt and it can generate a lot of information. That information could be. Oh, yeah, that's really.

We're good to go again.

Okay, sounds good. Let me go back a slide. There we go. Here we go. So we're in this world of generative AI, right? So you take a prompt and it's typically a text prompt, which may change in the future, and you want to generate information, and that information could be high volumes of text, which could be a blog, a contract, you know, how are you going to respond to a customer? But that also could be an image, a video. You know, I think we're just in the early innings of what it means to generate or have Gen AI. But what I actually want to talk about today is what we don't talk a ton about in this market. Given that all the excitement with like Mid Journey and Character and lots of on the consumer side is what I call synth AI. How do we take an enormous amount of information and synthesize it down to insights, which I actually think in the B2B and SaaS world is actually how us as decision makers actually can make informed decisions. First, let's talk a little bit about the differences between the consumer and enterprise world. I think we've seen a tremendous amount of innovation for generative AI, particularly on the consumer side. And so we've seen Mid Journey. Now you can create super fun creative images, right? In the design world you can create really fun videos. It's all about a big part of, and I'll be a little bit frank, a big percentage of traffic for generative AI today is a little bit more salacious, right? Some things are a little bit more on the porn related side. And that's actually likened to everything that happens in the Internet. Think about when the Internet first came about, a large percentage of the traffic was more salacious and more related to porn. We see that same thing happening with AI right now. And so on the consumer side, a lot of it is related to having fun, doing something new, creating, being innovative. But on the B2B side, as we all know, we have different objectives, right? We care about speed, we care about cost, and we care about quality. And so you can imagine if you are talking to your customers, you really care. Any communication with your customer is highly accurate. And so judgment and domain expertise is really, really important. And the copilot approach of having a human involved is really important. But as we think about how AI impacts the B2B world, we think about Gen is pushing out the frontier, the quality where we expect even better results actually for the new generation of AI apps. And we talk a lot about the cost benefit analysis of like disrupting workflow. It's a big debate right now of like, hey, is it going to, is the next wave of AI companies going to be workflow software company that embeds AI intelligence smartly or is it going to be AI an AI native first company that can get us to 90% of the answer and then you Embed some of the workflow and repository capabilities that make you a software business. And that's been the constant debate. I think what we, I think that will differ by market. But what we do know is that we're much better off by having humans in the loop today because what we really care about is accuracy, right? Domain accuracy, domain expertise and accuracy matters the most. And so we do believe in the copilot approach. As we, as we continue down this path, another thought to think through is, you know, again back in the kind of the copilot concept like we do believe AI in the near term for the vast majority of, I would say white collar jobs is to augment and improve humans. And here we've got a great quote from Barry at Hex, who's one of our portfolio companies. You know, they enable data scientists and analysts to, you know, in a notebook do incredible analyses. They leverage AI and able to do that. It still requires a human to actually make those decisions. And we do believe in the vast majority of near term B2B use cases A human will be involved in the copilot. And so let's talk about how we think about synth AI and its effectiveness. So I'm a framework type person, so I always like to think about things in a grid. We think a lot about the best use cases for synth AI. So think about condensing a high volume information down to an insight which is really what we think about in the B2B world. What you want is a high signal to noise ratio and a high information volume. The easiest example has always been customer support. I think consensus is that categories like customer support will be an incredible application for LLMs and AI because obviously there's a high volume tickets and there's a high signal to noise. And so you can use a copilot or deflection approach in order to remediate or answer a meaningful percentage of those tickets. Now the debate that we're going to have, which I'm going to continue on later in this presentation, is where will value accrue in a company? Will the existing ticketing systems, let's call it a Zendesk or intercom, embed AI smartly and capture the value in the space? Or will it be new startups who can meaningfully adopt AI faster, have better deflection and then build into the workflow when in this space. And I think it's a hotly contested debate which even we debate internally quite a bit. But I think the part that none of us doubt is that markets like customer support will actually accrue the most value and have the most uplift initially in the next couple of years from alarms and AI. But it's unclear which startups will get the most value. I think what we don't believe will be as effective in synth AI is something as simple as like, you know, think about like in legal AI which is obviously very hot topic right now given the amount of startups that emerge in this space. You know, searching and dissecting and augmenting a contract. So super high value, but actually negotiating a complex merger or a term sheet that is actually very difficult for AI and LMS to do. The last 5, 10, 20% of human decisioning because that is where actually the copilot approach really comes in. And so we're a big believer in using this type of framework as we think about where to spend time, particularly from an investment standpoint. So if you directly compare, we tried to give a couple examples of a couple ways to think about it. You know, I think marketing has probably been the category that has seen the most generative AI in the last 12 months. And so think about companies like Jasper and Rytr and Copy that have all grown tremendously fast in the last year, particularly around generating copywriting from a text prompt to generating blog posts and content and things like that. And it just shows the power of how great generative AI is on drafting in the creative phase. But what we get really excited about is the next wave of say in marketing which is personalizing that content for individual customers based off a high volume information which could be how they interact on websites, product engagement, data enrichment from third parties. I think the world is the oasis here in terms of how much information you can gather and now dissect and make informed insightful decisions when then powers actions. I think that's what's going to be really impactful about synthesis AI or synth AI is driving insights and then also empowering an action which is really the power of AI that you can now automate a big component of this and that's how you transition from being from the co pilot phase more and more to the agent phase. And so you can kind of click through some of the examples here. Legal again is one that we talk a ton about. You know, I would probably argue that we're not quite in the world yet where the average lawyer will say I want to text a contract because the reality is is that there's a lot of nuance in each contract. You actually a template sometimes is better than a newly generated contract. And so generate may not always be the right use case, but synthesize is tremendous for, for the legal space as you have high volumes of conversation of content. You know, I'm investing a company called Macro that's trying to reinvent, you know, what I would call is a document. And so the belief is that why is a PDF and a Word doc from two different major tech companies? In actuality, all documents should be API, indexable, searchable, just like a website. And because they're so good at embedding content in search, one of the major use cases is legal. And that's also what's really worked well for them in terms of new AI features is the ability to synthesize the key insights, understanding the clauses that you have to change in contracts and then also making the right recommendations. Because as we all know, winning a major, you know, lawsuit or case in legal can result in millions of dollars. And so the accuracy and ability to be correct and insightful is incredibly valuable. And so yeah, we talked a lot about the battle of workflow. You know, I think, you know, I might even reframe this if I was to say really what the battle workflow means is will modern incumbents embed AI and be very, very successful? And is this the age where software companies that were built in the last five to 10, maybe 12 years are going to exert their dominance even more? I think being modern is important because you have to learn how to embed the most recent LLMs, learn how to fine tune them, build out your own modern AI stack, which may include a vector database, which may include model hosting, which may include actually going all the way to open source foundation models, or will be a new up and comer. I think this is probably the key debate. I think we've seen a number of modern incumbents do really, really well. Notion is a great example. Notion AI is probably one of the fastest growing AI businesses out there, but obviously an existing product and like they've been really smart about embedding smart agents, smart tech summarization, copywriting, and they own your notes, your repository. They have so much more information to work with. You've laid out a couple other examples here. Zoom Info recently integrated GPT as well. And you can imagine the amount of sales enrichment data that Zoom Info has to then help power lead scoring, outreach to customers. All of their applications actually have a huge advantage. And so we debate a lot internally like do we go after modern incumbents or do we go after the next startup? I think the next startup has a huge advantage. And you know, I don't know how many early stage founders we have on the call, but I think where the next, you know, early stage founder has a huge advantage is figure out a way to build your own proprietary data. You know, I'm investing in a company called Pave and they didn't use AI to build their data mode, but like their data mode is what makes the company so successful. So they, they are a compensation company. And what they decided three years ago was isn't it crazy that you never know how much someone should be compensated for any employee? Right. The whole world is hiring people, you never know how much to pay them. And the way that you find out how much the average person gets paid or gets comp benchmarking is you go pay tens of thousands of dollars to a company like Bradford and they run an annual survey and maybe in two months you will get this really, really long output report of how much someone should get paid. And the reality is all that data is in your HR systems. And so what Pave said is I'm going to API my way into a give get data share model. So if you share as an HR leader your compensation data, you will forever get benchmarking compensation data for free. And then Pave builds all these software tools on top to manage compensation, do merit cycles, all that kind of stuff. Now building a data moat dramatically enables your ability to win on more than ease of use and go to market execution. And I think one of our theses is that AI, especially for new companies and founders, is an incredibly fast way to build a data moat. Right. What you don't want to be in is a situation where only a wrapper on top of say GPT4 and you're just 90% of your business is calling GPT4. But if you are leveraging GPT4 to help make informed decisions and build a unique data mode of how customers respond, how contracts are formed, how do users like to engage with your product that suddenly becomes a really unique asset that you can leverage to grow and become a lot larger and compete against all these modern incumbents. And so with that I might open it up with questions because I'm sure there's lots of debate here. I see the meeting chat exploding, so maybe I'll hand it over to Caitlin and let you drive any questions that we may have.

Yeah, of course. Thank you so much for that and love the really detailed examples as well. Our first question will come from Josh. If you want to unmute yourself and ask Josh, Josh, last names, man, they're so difficult. That was butchered I'm so sorry, Josh. If you do want to unmute yourself and ask.

Hi there.

Hi. Do you want to say your last name so we can all make fun of me together?

Shachter. Is that what you guessed?

Oh, no, not at all. But thank you so much for correcting.

So, yeah. So my question is around kind of what you're thinking of as far as appetite for investment into gen AI products and solutions as we go into 2024. Is it really just the LLMs and the foundational level that are going to get all this gunpowder, or is it these app layers around the workflow?

It's a great question. So I would say probably the front half of this year, the battleground for figuring out what does the infra AI stack look like, was at play. And so I do think there was a tremendous amount of venture capital dollars that chased that opportunity, but now I think actually we have a decent understanding and there'll be some evolution. Right. But there's going to be foundation models and there's going to be closed ones and there's going to be open source ones. Right, Right. And I think that, you know, that is an amazing opportunity for app companies to now build their infrastructure, because I think the infrastructure layer had to first get built out. So we've got the foundation models, we've got vector databases, we've got prompt engineering. You know, I still think there's going to be some version of an analytics play to actually measure efficacy at the infra level. And there's model hosters. Right. And there probably will be even more categories that form, but those five categories alone now set up a really solid infrastructure layer for any app company to move really, really quickly.

I have a quick follow up on that, if you don't mind. We've all learned over the past 18 months to be very disciplined with our budgets. Right. We're not in 2020 anymore, but. But AI is expensive, right? The gen AI is expensive. So as an investor, how do you think about that? Should the companies that are investing in IT to use in their platforms? Should they. They plan on Moore's Law, Right. Is should they plan on a rapid decay in the price of it, or should we be more conservative with it because, you know, the rates might stick around?

So I do think we believe in Moore's Law and we're seeing the costs amortized dramatically. And quite frankly, venture dollars are massively subsidizing a lot of the GPU costs, particularly at the infra layer right now as they're doing inference. But can I Ask Ashley what does update do?

Oh, I would love for you to ask what update does. So simplest stated, we are gong for customer sex. Sex customer success and post sales conversations with customers. And we actually play more into the synth AI because we're synthesizing all the conversations.

Awesome. Okay, so that's helpful. So I, I think from a cost perspective, right, because you're dealing probably actually with a higher volume of data given that you're ingesting zoom calls. So you do have to factor in. I think that's actually where the infrastructure layer helps you quite a bit. Because part of the benefit of some of the new platforms like we're investors in Pinecone for example, is that like you actually can do a little bit of fine tuning and then optimize the amount of context that you actually pass through the context window for a foundation model which dramatically reduce costs. And you're also in a category where I would say from a pricing standpoint, you know, we've seen a lot of pricing innovation on the app layer in terms of do you charge based off credits? Do you bundle the pricing into the per seat pricing? You know, I've seen a huge mixture of both, but I would say the trend has been for more PLG or self service businesses. They've tended to go more after they've had more success doing perceived pricing because as you can imagine the.

I'm sorry, what's perceived pricing?

Percy? So bundling. So let's say you're seat.

Oh, per se.

Yeah. So let's say you originally cost 20 bucks per seat. Now with AI you're it's now 30 bucks per seat for that license or you add an extra $10. And the reason why that's effective for more PLG or self service businesses is the reality is like 20% of your users will be unprofitable, but 80% will be super high margin. And so the more self service your business is, the more likely you're going to go to per seat pricing of embedding the AI pricing in the more enterprise you are, I've seen more people have usage based tiering pricing and the enterprise will be more accepting of that.

Thank you.

That's just a general trend I've seen. But the short answer is I think right now, because the infra layer is finally set, we are seeing a tremendous amount of innovation at the application layer. And I think the real debate is not whether AI will find its way into most app companies, it's more how do you find differentiation when a large percentage of companies initially are doing an API call to say GPT4 and need to find other ways to differentiate other than having a better AI response.

All right, thanks for that question, Josh. Our next question comes from Foley 8733.

That's just like a personality test. But my question, yeah, so I have a bunch of them, my name, our company loves to do that sort of stuff. My question is when about like data moats. And I kind of feel that everyone, when you're in your own little sphere thinks you have great data. But from your perspective, you see so many, you know, across your portfolio. What are like KPIs that you guys use to say, hey, your data mode is good enough where you should build a product around it versus kind of falling into the trap where you build this product around your data and then you realize it's not useful to, you know, a critical mass call it. Then you just kind of, you know, put all this effort and money into something that's not really going to gain any traction.

That's a good question. So it's a little bit hard to have KPIs around data modes because it depends on what type of data you're collecting. Right? So I don't think there's a volume metric or like it's very hard. Like if you're collecting, I don't know, like take update for last. Like if you're collecting customer call information, like, the comparison is probably going to be more how does that compare to other call tracking or call recording companies and how much customer conversations they have? And so it's a little bit hard. I will say I tend to like to use the superhuman definition of like, product market fit to everything. And I think it applies to data modes as well, which is if you suddenly shut it off, how much of your customers would be outraged? That's actually the real test of a data mode, right? Because like, if your customers are like, you know, that's kind of annoying, but I can export it and move it somewhere else, then the realistic answer is that you don't have a super strong data mode, but you might have other modes that might be like, related to the workflow of the repository. But to truly have a data mode, the answer has to be you shut it off. Your product doesn't work like a CEO is going to call you instantaneously, say, turn this back on. And so probably not the answer you're looking for, but unfortunately it's very hard across markets.

Thank you.

All right, if we want to put down the slides too so we can see some people Better as we're answering these questions. Awesome. Oh my God, so many wonderful faces. Our next question comes from Steve. If you want to unmute yourself and ask.

Yeah, my simple question was what's going.

To be the global impact of AI on electricity usage and is that going.

To then have a macro impact on.

The environment and climate? And what's Andreessen Horowitz's view on that?

Ooh, that is a hard question. I will be honest, I do not have a strong opinion on that, nor do I think we have a firm opinion. What I will say is it's very easy to get excited about AI these days. It is a massive platform shift and what is possible, it's just so tremendously different. If you look at every platform shift, it takes at least five years to actually see what the mega global impacts are. When the Internet first arise, the reality is half the enterprise blocked was mostly porn and people didn't know what to do with it. And only five plus years later did new markets arise. And mobile is maybe the more prevalent example. When mobile first came about, we all had mobile devices. When did they come out? I forget exactly 0708, somewhere in that time period where it's mass production mobile device, everyone had one. If you want to talk about what markets emerged most prevalently with mobile devices. Rideshares actually. And like the sharing economy probably was the newest market that no one ever even realized that you would have a computer in your pocket. And it was a combination of mobile and payments that enabled a whole new paradigm of how we purchase, how we ride, you know, Instacart, Swim, public, you know, like that's a great example of like that company could not have existed to have people pick your groceries as well as have people order at home and have the entire infrastructure to do it. And so I'm not answering your question whatsoever at all, but I think the short answer is we have no idea. Transformers have been in the works for years, but only through mass adoption starting in less than the last nine to 12 months. I think we are years away from actually knowing global impact yet.

Got it. Thank you.

Yeah. So Steve, great question. If nobody knows. All right, our next question comes from Alina.

Hi Christina, thank you for the presentation.

Actually, my plans through Adam.

My question is, do you.

Are you aware of the financials behind running LLMs right now? I was reading in one article that.

The startups that are running LLMs are estimating that the cost per user is.

Around 100,000 a year. Do you have any data to share on the cost it's really cost, it's so I think it's very use case dependent, right? So I would say like if you take, if you're just dealing with small form text, right? Like if you're dealing with hey, I want to call GPT4 and I want a response for a customer and the response is an email that's actually not a tremendous amount of text that you're passing through and that's actually quite cheap, like in the, you know, cents category per email. Right. And so that then it depended on like how many emails are people sending, how much you're going to cover the costs versus your end customer. But if you are generating videos, for example, you know, I would say videos, even a 5 second clip is probably costing a couple dollars. And you can imagine every video you generate, you're not just going to generate one, you're going to generate dozens because you're actually iterating and editing over time. And so you can imagine that cost as like the duration of the video gets longer and the amount of times that you want to regenerate to get exactly what you want is actually quite expensive. Right? And like it scales like more over time. And so it's very dependent on the amount of data that you're passing through the context window and how many times you're going to have to generate. And then it's also dependent on are you going to leverage an existing foundation model like GPT4 or are you going to build on your own open source model, maybe using llama of which you have to start factoring GPUs and inference costs which is significantly more expensive. Right. And I think there's a heavy debate as to what is the right approach. I don't believe there's one right approach. I think every company is going to have to figure out the economic trade off and the differentiation factor of doing it in house. And then obviously there's the option of entirely building your own foundation model which I think at this point is going to cost hundreds of millions given the precedent of like as you look at like OpenAI, anthropic and even the more specialized models as well. And so I think it is an economic trade off as well as a differentiated trade off by market. So it's very dependent on what you want to build.

Any follow up to that?

All right, thank you.

Of course. Our next question comes from Ajay D.

I don't have a question, I just put a comment.

Oh, okay. Sorry about that. Okay, Our next question is what do you think about AI councils? How do you think they will be most effective in the future?

Can, can the person elaborate more on that, what they mean by AI councils.

Governing bodies that some organizations are putting in for ethical like making sure ethics etc are withheld with the addition of AI in different organizations.

Got it. I mean maybe the most public form of this is how involved will the US government get in AI regulation and is that a good or bad thing? And you know, I think it's hard, right? Like I think the reality is is that, you know, there's been debates as to should we stop all research until a framework comes about and I think that would be a huge shame because then you're stopping innovation without the opportunity to co create regulation and frameworks for protection. And there's no doubt that there are going to be bad actors. The reality is OpenAI has not released some features because there's fear of bad actors taking advantage of it and I think there's going to be constant scenarios of that. And so do I believe that there needs to be some form of regulating body that will help create frameworks for protection for AI? I do. I think some of the recommendations of how aggressive or to pause all innovation until you figure it out I think would be unfortunate. So it's an early debate, I think knowing exactly how to regulate AI and agents I think is very, very hard. What has pleasantly surprised me is this is not the first AI wave. It's just the first wave where Transformers have gone to mass adoption. As I think about prior AI waves, even from five to seven years ago, I actually think the average consumer sentiment is significantly more positive on AI and its possibilities than AI and its fears. And so I actually view this wave as exceeding my expectations of fears of robots taking over an AGI and all those kind of like movie type concerns. Because I do think the world has gone through multiple waves of this and this time around I actually think there's more desire for the upside than the downside that I've seen in prior waves.

Thank you for the insight, Ahmed. Do you want to ask our next question?

Yes, I'd be happy to. I posted also on chat. Great to be here with Christina and other folks from Saster. My question has to do with emerging AI companies in the areas of customer service, support and success. What companies are you seeing there that are promising and do you see the use cases mostly around customer engagement, agent copilot, or perhaps even managing the life cycle of the knowledge or content within those operations?

So I think we're seeing a lot of innovation on the customer support and success side, it's still early. I think what I've been very impressed by is actually I think the, I call the more modern incumbents, like someone like an Intercom has just been really aggressive in AI, you know, like I think they, if you look at their most recent products, like a lot of the ticketing systems have an inherent advantage because they one have a lot of data and they already have the distribution with the customers. And so companies and ticketing products like Intercom have been really aggressive in releasing their LM enabled chatbots to do deflection and have done very, very well from that standpoint. I think, you know, I always look at the YC class as like a indication of where early stage founders have interest. And I want to say like, I wouldn't be surprised if like 15% of the YC class was related to customer service, customer success, something related to AI in optimizing responses to customers. I think most people are starting with deflection because it's perhaps the thing that customer support talks about the most because it's most ROI driven. And also I think ChatGPT trained us to interact in a chat interface and so I think it's easier use case to imagine. But it's a little bit unseen as to who's going to win the market. But it's probably an area that I've seen probably the highest volume of new innovators and companies emerge.

All right, our next question comes from Pete.

I. Christina, apologies, I'm in a strange room so I won't be able to come on camera. Hopefully everyone can see my face and hear me. So I'm in a space. I'm a founder of a company called Experiential and we actually went the data mode route before it was really fashionable. And I remember early in the year most people were saying leverage chatgpt. Leverage chatgpt. But our customer audience would find that information not relevant because, you know, they're skilled people who know what they're looking for. So my question to you is, are you seeing now an emphasis from investors with alignment of the founder if they say they have a data moat, if they have a relevant background directly within that data mode domain? Expertise. Right, right. So they can validate whether that data is going to be useful to their customers or not. Are you seeing that being more of an emphasis? Is Andreessen doing that?

I mean, I don't want to speak for all investors. I think I personally believe in a data moat. And so now what a data moat means I think dramatically differs to Everyone, to me what it means is there's some set of proprietary data that if the customer, if you shut it off one day, the customer can't run their business. And so like there are obviously many different versions. And it's also how you build that data mode, right? Like is it, you know, is it first party or third party data? Right. Are you acquiring it and merging together? That's I would argue a weaker data moat than building first party data. And so there's a lot of complexities in like what a data mode actually means. I think it also depends on how you leverage the data mode. Right. And so the reality is I'm still an old school software person. I'm just a big believer that you have to own the system of record or the repository in some way and you have to be holding onto information or data in a structured way that if you shut it off, the customer can't run their business. And normally that is tied to the data mode. And so I think what has been the most challenge for a lot of AI app companies is they've grown tremendously because consumers are so excited. Consumers and business users as well are so excited to get access to some of ChatGPT's capabilities and responses that effectively you get. It's easy as a founder to get wrapped up in the tremendous growth and then you become a Wrapper company around ChatGPT or GPT4 and that becomes really, really hard because 10 companies go do it and then your economics suddenly decline. And we've seen that happen in a number of AI apps already. And so you do have to figure out a way to make sure there's either a data mode, a repository, a system of record that like enables. And by the way, like, I'm a stickler for KPIs. Like the easiest way to see it is just retention if people are using products right? Like not every category means being a dau products, but being a DU product means your system of record, like that is a big part of it. And then two is like, do people keep paying you? I've seen retention just like nosedive in the last six to 12 months of companies are growing really, really fast. And because you have lots of people who are really excited to try your product and that's awesome. Top of funnel. But you also have to make sure that like there's some base that like really actually needs and is dependent on your product in order to be sticky.

Dan, a follow up to that question. How are you seeing the strategy of, especially if you're doing enterprise B2B launching on a marketplace, basically piggybacking on an existing app on top of having a data mope as a way to get your distribution. So instead of going direct sales, but being a marketplace, especially in this AI.

State, I think that's probably a comment separate from AI, but just like how do you think about go to market distribution? And I believe that leveraging marketplaces can be phenomenal to get your awareness and brand out there, but it should never be the majority of your distribution. Like ultimately you have to control your own destiny. And so whatever direct means for you, that could mean self serve, that could mean plg, that could be outbound sales, that could be inbound sales, the majority of your customers and revenue, you need to control your destiny and be able to own that. But leveraging channels via marketplaces, via affiliates, you know, I think all that is fantastic and great boosters, but you just don't want it to be your whole business because then you have massive platform dependency. And you know, the reality is, is that like the marketplaces that you play in, they have their own incentive models and you just don't want to be in a situation where you get screwed.

Awesome. Thank you.

Ahmed, did you want to jump in on the conversation here?

Yeah, I would love to give a follow up example and I really like the presentation where you mentioned the incumbents. Typically they look at their workflows that they have created and look at AI as an appendage to their workflow, perhaps to automate certain stages of the workflow. And it's part of also innovator dilemma that they have the workflows already built into their applications. Whereas startups have have seen actually when they start with AI, they come up with the new workflows that sometimes then forces the users to use their product as opposed to for example the underlying system of engagement which are typically Salesforce. And I do remember from my previous company Support Logic, we had a escalation prediction AI based escalation prediction module and application and our users are people who were managing escalations and they had workflow for management. But for the predictions and what to do we had to create a separate workflow, but that was outside of the underlying CRM, be it Zendesk or Salesforce and therefore they wanted to see it back in their product. So I think that that dilemma is there. Would you actually create a new workflow that is AI induced? And probably those we've done primarily by startups or you look at the opportunities for automating certain aspect of workflow within the incumbents CRMs, is that what you're seeing, that sort of innovator dilemma for companies like Salesforce and so forth, so that they don't necessarily go way out in terms of identifying new use cases that would maybe even obliterate their existing workloads?

Yeah, I mean workflows. So this is the debate that we have constantly internally is historically, before AI, like when we were focused on evaluating software companies, you cared about being a system of record, so you want to be the repository and then you cared about owning the workflow. Right. The more real estate that you took up of a user's time normally measured as a daunt or percentage of time spent, the better. Now with AI it is uncle, it's clear that AI is going to eat into the workflow. Because the reality is you should be automating a big percentage of the manual tasks that happen in Salesforce, for example. And the question that we still don't know the answer to is so we take Salesforce an example, like do you just believe a meaningful percentage gets automated away and so there's no workflow? Right. In aggregate, AI should massively decrease the amount of time spent on workflow, or do you believe there's a new workflow interface and it pushes Salesforce further down, deeper into being the back end system of record and you create a new version of a workflow that it's reinvented. I don't think we know the answer to it. I think this is going to be very market dependent. Another thing that we debate internally is like if you really think about a system of record has really an old school definition which is we moved an Oracle CRM from on premise into the cloud. Fundamentally it's a database with a rigid grid of rows and columns of how you store structured data. And we had a massive platform shift of moving it on premise to the cloud so everyone can use it. Right. And then SaaS happened and the belief was that everything's going to be best of breed and so everyone's going to build 80% of what you need in the broader system. Well, how do you think about the next version of that? Do you generate new system of records dependent on what each customer or user needs? Because the reality is, the reason why there's so many different SaaS companies in any category is everyone's 80% is different. And so it's created an opportunity to be best of breed for a bunch of subcategories. So the question becomes, does a new system of record become the ability to generate a dynamic system of record that is custom fit to each user. Who knows? I think we're still in the early days of actually understanding what I do think the definition of system record is going to change. Maybe that's my main thing. And I do think owning the workflow and thinking about being a DAU product is going to change now with AI and automation and how that transforms yet we kind of don't know yet.

Thank you.

I told you this audience is super interactive. Question after question. Our next question comes from Mohamed.

Alrighty, Christina, first things first. Thank you so much for your time. My introduction to venture capital.

Thank you.

Goes back to 2019 when I got introduced to this book the Secrets of Sandhill Road by my mentor John Gleason. And it's absolutely exciting to be here now finally listening to somebody who represents the same space. My question to you before I go ahead. I'm the founder of a company called Remote Bridge. We help early stage startup companies get to work with best in class talent overseas to improve unit economics. We're moving towards building an MVP that would include some sort of gen AI, but we are still not there. My question to you is from an investor's lens, how do you assess the trade off between simplicity and complexity when picking and choosing the right apps to invest? Where does that break essentially? I do have a follow up question as well, but I'll pause right there.

The question is how do we evaluate simplicity versus complexity when choosing to invest.

Gen apps that are on your radar?

I almost would just say it's probably not the right framework. The way that I don't want to say all investors, but the way that I think is is there a driver for something that exists in a market? I go back to the PAVE example, right? With like in the compensation phase, you know, There was like 20 compensation companies at the time that we invested in Pave. What does that tell me? That tells me that well there's clearly demand, right? Because if I talk to 10 CHROs, they would tell me that compensation is a big pain point. And when 20 companies emerge in a space, to me that signal and demand that everyone has recognized that pain point. And then the very hard task because the worst thing as an investor is to invest in the wrong player, right? Especially as a big fund. The next task for me is to hypothesize what's going to be that unfair advantage for a company to win. And with pave, that unfair advantage was the belief that they were going to build the largest data mode of compensation data. And by the way, they were not the biggest company. So it wasn't traction. They were not actually necessarily on paper the domain expertise and compensation. They were much more HR leaders and people who are from industry building the company. But like as the hard job as an investor is to have a thesis and like the way I always articulate my thesis is like, what's the unfair advantage to win? And for pave, the unfair advantage win was that they were going to collect all the data. And if you think about every ATS company, so applicant tracking company in the space, right? It's been a tough space and people don't think about LinkedIn this way. But LinkedIn is the biggest applicant tracking company in the world and it's because they own the largest database of online resumes. And so like I think that that is actually the lens that I think and every investor has a different way of thinking. And so some markets are going to be about the complexity of the product and how that differentiates you and some markets are going to be about how quickly and easily you can get a customer into your product. Right. Like as I think about, you know, I think about Grammarly is always an interesting example. Grammarly is not a complex product, right? Like it's a Chrome extension that enables you to correct your grammar very simply and easily. And like you would never have said, oh, I'm going to invest in the most complex product in that space. Because the reality is there probably were very complex products that enable you to correct your grammar in more meaningful ways with dedicated apps and all that kind of stuff. But the beauty of Grammarly was that they really understood their customer. And actually what most people don't realize is like they got started with a bunch of people like my parents who were non native English speakers in the US who really wanted to show well in their emails. And they grew tremendously actually through the Chrome Extension store at a time when that store was growing very, very quickly and use that as a jumpstart to get started. And so that is a simplistic product. But I don't think anyone who invested in Grammarly would say, oh, we invested in it because it was simplistic. They had an unfair advantage of distribution, understood their users of like who was going to catalyze them in the early days. And so that's probably the lens I would take.

Gotcha. Yeah, thanks for sharing, Katelyn. If there are other questions that people want to ask, I'd probably just pause so that everybody else has a chance. If we still do have a minute, I'll jump right back in. But Christina, thank you so much.

Yeah, we have time for one more question. Thank you for giving the ability for other people to ask. So I'm going to read the last question aloud on behalf of Joshua. There is an emerging trend of AI driven browser plugins that can operate on and analyze data directly within the browser. I could even imagine an AI browser coming along given that a significant amount of data is exposed in a browser. Do you think these AI plugins have the potential to challenge or even displace AI driven SaaS services, especially those that, that rely on proprietary data moats?

Super interesting question. We spend, I would say we spend a tremendous amount of time on browser Chrome extension plugin companies right now. It has definitely been a huge trend and there's huge pros and cons, right? There's the pro, which is it's very easy to adopt and you can capture a tremendous amount of information. The con is everybody gets the same data. And so there is a real question of how do you build a defensible moat when everybody is getting the same data? We did make an investment in one company, which is actually not a browser extension, but a company called Rewind, which is a Mac app that enables you to screen record anything natively on your computer. So it functions like ocr. So if you see it on your computer screen, Rewind captures it. And what they're trying to do is be AI for memory, because if you get old like me, you know you saw the revenue number for X company somewhere. But I can't for the life of me remember if it was on Slack, in a Zoom conversation, in my email or any of the other 10 chat apps I use. But if I saw it on my screen, Rewind will find it. And so it's locally stored screen recording, which is what we viewed as one way of building a data mode. I think the big challenge of the Chrome extension companies is not the fact that they're Chrome extensions, because they actually again, don't necessarily believe that owning the workflow and the real estate is going to be as important in this next wave of AI, which is the historical challenge with Chrome extensions. But the bigger challenge is how do you actually have a differentiated mode? Because everyone is capturing the exact same data. And so if you are capturing DOM data and product usage data and user engagement data, you think it's a moat. But the challenge is literally everyone else has the exact same data. And so how do you then mirror that, merge that data with something else, or get the user to do specific things that suddenly becomes your data mode proprietary data. That's going to be the big challenge in the space.

Amazing. We're at time. So appreciate you, Christina, and so appreciate all of you for asking so many wonderful questions. Really helps the conversation. So thank you to everyone for all being teachers on this session. Christina, wonderful session. So many engaging chat conversations and examples. Thank you so much for joining us and we will be back next week.

Well, thank you everyone. And Caitlin, thank you so much for hosting.