How Enterprise Companies are Buying AI (or Not) with ContextualAI, Anthropic, and Glean

Hello everyone.

Welcome to Sasta's Enterprise AI panel. I'm Sandhya Hegde. I'm a partner at Unusual Ventures, which is an early stage fund exclusively focused on enterprise software. And one of the things I have been doing is working with startups that are trying to figure out how to sell generative AI software to some of the biggest companies in the world. So really excited about the topic and would love to kick off by having our really incredible panelists introduce themselves.

Hi, my name is Arvind. I'm the CEO of Glean. Think of Glean like a. It's a search engine for work, so think of it like Google or ChatGPT, but for your business. Really excited to be here.

I'm Ben. I'm one of the co founders of Anthropic, which is an AI safety research and product company. We have partnerships with Google and Amazon to sell our models to all of you lovely people. We hope to be the safest and best models out there.

Hi, I'm Dao, CEO at Contextual. I'm also an adjunct professor at Stanford. Before this I was the head of research at Hugging Face. So this new company, Contextual, is about enterprise large language models. So how do we get this technology to be really ready for enterprise deployments? We think language models are great first generation technology, but they're not quite ready. We see a lot of frustration in the market around that and we're trying to solve those problems.

The hook of this discussion, and in my mind, the way I think about it, as early stage investors, we're always looking for what is new technology and where is it in the adoption cycle in the world. Right. Has it crossed the chasm? And while GPT has become a household acronym, there are probably over 200 million people in the world who have at least signed up for some generative AI software. The reality is that some of the big companies in the world are very early in their adoption. Right. Perhaps Google has started rolling out generative AI. Sure, but what about Walmart and JP Morgan? And so that's the big discussion today is taking advantage of the incredible wealth of experience and perspective we have here on this panel to understand how are these big companies thinking about leveraging AI? Where are they in their buying cycle? What are the barriers to adoption for them? What can we learn from people who are already taking steps making these investments? And last but not the least, some predictions for where we might be in four or five years. Like what would still be really difficult and what might be more achievable by then. So with that, I kick off with our first big question. What are big companies most excited about using generative AI for or modern AI for? Whether that's language models or diffusion models, modern speech synthesis, like what is it that in your experience, big companies are excited to use this for?

Yeah. As we work with enterprises, we've seen a level of excitement that I think has never been seen before. And so I would say that enterprises have two big themes. One, they want to use AI to improve the products that they sell to their customers. And second, they want to use AI to transform their business and how they work, their employees work. So some of the big use cases that we have seen are across customer support, like how do you actually make your agents be a lot more productive? Have AI automate the process of resolving a case or actually bringing AI right in front of your customers so that they can get quick answers to their questions. That's one of the big use cases. But also say that the use cases are actually across Personas, like for sales and marketing, like helping marketers to create content using AI, and for engineering, helping, like helping their developers write unit tests, testing code, as well as just general code troubleshoot issues using AI. Those are some of the big use cases that we've seen. Yeah.

Kind of building on what Arvind was saying. We've been really impressed at how even the largest companies in the world, and not just in tech like Salesforce, but also banks, retailers, et cetera, are moving with this technology. Where for example, one large bank that we were talking to, they came to us and they said, we've talked to everybody in our company and we have 500 different use cases that we want to apply large language models to. That's really incredible. And they don't even know where to start. So working with us to say, well, what are the capabilities of these models? What can they do today that's really high quality? And what might be more of a stretch, where do you think the biggest value add for our business will be? That's been really important. And then beyond that, how can they make AI the expert at their company, both in what they do as employees, as you were saying, but also an expert in what their product is so that their customers don't have to go learn about what the product is and read all their documentation, but instead the AI, they can just talk to the AI as if it was a solution architect or forward deployed engineer and be able to use the thing immediately.

So I was saying just backstage to Ben that we probably talked to the same bank so there's a lot of excitement everywhere. And so you can really feel it, like everybody just knows that this technology is going to change the world. But at the same time, I think you can see in a lot of these enterprises that people aren't really sure how that's going to happen. So there's a lot of excitement, but what it's actually going to look like, I think we don't know yet. So we are trying to discover this. It's all a very experimental phase. We're doing a lot of piloting, trying to figure out where exactly is this technology going to matter the most and where is it already ready and where is it going to be ready soon? And so I think if you look at the landscape of use cases that we see right now, there are roughly three big buckets. So one is around information discovery and information synthesis. So companies like Lean, I think, are doing a really good job at that. But there's this whole synthesis problem. How do you get deeper insight? So not just the data that's there, but the information you can derive from that data. Then there's a lot of hierarchical summarization use cases. So how do I really turn it into something I can act on? And then there's a lot of support chatbots, but just chatbots in general, where you have some script that you want to follow. So I would say 95% of all use cases we hear fall in one of these buckets. There's a bunch of other exciting stuff too, but I think those are really the main things. But then within those buckets, I think many companies are still trying to figure out what exactly they want to do. So, exciting times.

I'm curious, when you are advising these folks on what are the right use cases to focus on, how do you help them understand what are the use cases that language models tackle extremely well enough that an enterprise can adopt it successfully, versus what are use cases where they should probably look again in 12 months, two years, as opposed to try to make it happen now.

So for me, the best use case is one where you can define what success looks like. And we see surprisingly few of those kinds of use cases, actually. So this technology is great. Like I want to try it on my chatbot. But when we ask people, how do you define success, how are you going to measure that this thing is actually good enough for a production deployment? Very often they don't have a good answer. That's really one of the things that we're looking for first is just do you actually understand what you want we.

Often help customers along to get to that stage where we say, here's what we've seen work for other people, here's how you can construct an evaluation set where you can continuously improve. That's where we're trying to push, especially the large companies that have let this slip out of their DNA and not only figuring out how do they try things and measure themselves, but also try a bunch of things. So take a portfolio approach. Try some less risky use cases like where knowledge extraction or summarization might be involved and maybe some more risky use cases like tool use, where it's using that company's tooling function.

Calling makes sense. And now maybe coming to the meat of the topic, what do you see as the biggest barriers to adoption in enterprise? Whether that's like just at the pilot stage or for a company wide rollout. And there are a lot that have been brought up in the general ecosystem from data governance to reliability, security, safety. But I would love we could go a few levels down and share some examples of what are the things that have held up sales conversations and product rollouts for you where people really wanted to adopt something and got stuck. And where are they typically stuck?

One thing that actually when the market is so early, everybody's confused. Like people actually have this fear of, okay, like I'm seeing this tool right now from you and it looks good, but what if there's something else that's better than this? And so I think that actually holds up sales for us for sure sometimes where like customers want to do a lot of research. There are hundreds of companies building hundreds of products and they won't actually go see it. So that's just a basic limitations of sales process. And I think one other thing which I would also just add, so security has been big, but most of the security questions, at least in the media, have focused on companies are worried about having their data leave their environments and going to the model companies like Anthropic. And then your proprietary knowledge actually becomes public knowledge at that point being trained.

For the next generation.

So that's been like the focus of worry. But there's also like a big issue internally. If you look at any enterprise, we have lots of knowledge and that knowledge is actually not available to every single person in the enterprise. The governance is actually a big part internally. And so when you have these like tens of different AI tools coming into the company, all of them need to work with knowledge, your enterprise knowledge. And so if you start to just copy your enterprise knowledge from wherever it exists today into all These different AI tools, then you lose that governance. You have no idea. Now, is that information being used safely and only being delivered to people within the enterprise that had permissions to use it? So that's actually another big concern that companies are grappling through and figuring out should they have a more centralized sort of procurement strategy for AI tools.

Yeah. To underscore something Arvind said, we find that having Bedrock Amazon's foundation model hosting program available to our customers is extremely helpful. The most sensitive customers want things like Fedramp certification and things that take multiple years and a ton of effort to implement. And so when we can just send them there and they're like, oh, you already have everything, all the corners squared, the I's dotted and the t's crossed, that makes them much more comfortable. And those conversations are much easier. But even for those conversations, it's still a question, as Dao was saying, that we need to help them figure out how do they actually use these things, where often in their earnings call, their CEO will say, oh, we need to have an AI strategy. But underneath them, a lot of the departments are still thinking about, I have the OKRs that I had been planning from last year and I have all these maintenance and tech debt projects that I need to complete. And there's just not a lot of extra bandwidth in the organization to explore these brand new things and figuring out how it will transform their company and their products. So in, in our cases, for example, Notion, I think is a great example where we worked very closely with their CTO and everybody, all the way down to their engineers on the front lines to deeply integrate anthropics AI into the Notion product experience. And we think it's extremely good, but it needed a lot of dedication to make that happen. And I think a pithy way of summarizing this is a lot of people think about this new AI technology as something that's going to just come in and work from day one. But actually it turns out it's still software. And with software you need to do the work of doing the user research, iterating with your, all your different teams, from marketing to design to your engineering, and hopefully in the future it'll get easier. But that's where we are today.

And so for me, the main things that are really preventing mainstream adoption are, I think, very familiar themes to many of us. So hallucination, these models that make up things, we don't really know why or how we can fix that. These models have attribution problems. So you want to be able to trace it back to something in the training data. They have compliance issues, they can't really forget things easily or you can't really update them easily. They have data privacy issues and they don't have the right cost quality trade off for many enterprise deployments. So very often they're just way too expensive to be actually useful in enterprise setting. I think these things are all getting fixed and so I think all of us are trying to solve these problems and you can solve them in lots of different ways. And then once that happens that's really going to. What is it? Unleash the floodgates and then yeah, this stuff is going to be everywhere.

I'm curious, especially since you brought up notion as an example. Obviously a lot of the early adopters in the enterprise have been software enterprises. Right. So like software companies that are now big and have some of these problems, like they do care about data privacy and reliability, but they are early adopters of all good new technology. I'm curious kind of what patterns you are seeing in terms of what are the types of companies and who amongst. Who are the people in these companies who knock on your door saying hey, I need your help getting tools like this adopted? What have been some patterns that have worked well for you in terms of enterprise wide rollout?

I think the interest in generative AI is actually, I think it's universal and we're actually seeing companies that are still using mainframes for their technology, but they want to take that leap and go directly and embrace AI and be the number one adopters for AI mainframes to.

Generate a pretty big leapfrog.

Exactly.

And I think, and I think partly like folks are actually seeing, I think digital transformation has taken a lot of time for a lot of companies and companies that took longer, like they feel like this is the way where I don't want to be behind then in terms of like, just like who's like for us, the CIOs we're seeing a lot of interest from them in terms of adopting generative AI. Of course, like if you think about the role of a cio, they represent the demand coming in aggregate from the entire company. And with Generative App, that's how it is. Like salespeople want the technology, marketing people want it, HR folks want it, IT wanted for automation. And so I think the CIO has become like the national focal point for us at least to actually go and bring our product.

Yeah, I guess tying to the like more dinosaur type companies. We're talking to some of the largest banks in the world. And they're very small things, or maybe they seem small to us, that are critical for them being able to use this technology effectively. And that's just doing arithmetic. And so you could, like on in Unix there's bc and if our language models could use a calculator program, then they would suddenly be extremely useful for all these large companies. And the fact that tool use is still hard, I think is an open problem. And then also what Arvind and Dao were saying about hallucination, also very well known problems, companies like Perplexity are figuring this out of how to make not just citation work, but also in a user experience that people can understand. Because at the end of the day, you can't rely on your users to click through every single link and verify everything themselves. So we need to have some kind of deeper understanding of where is the language model confident? Where is the language model not confident? And I think once we've covered those things, we'll see more early adoption from a wider range of technology companies, not just the newer ones, the Silicon Valley ones, but everywhere in the world.

Yeah. So for us we're seeing the same interest, I think, that you two are seeing. And there's a lot of possibilities, I think, for working with interesting companies on interesting use cases. The early adopters, you tend to have very tech forward companies that are basically just ready to go, but very often they think they can do it in house. And so I think that belief is probably going to go away in the next couple of years where people realize that this stuff is a little bit more difficult than they initially thought. So aside from that, I think one of the interesting things that we're seeing is that there's really a mandate from the CEO down, whereas we have to do something. And so to me that's exciting because it's a business opportunity. It's also scary as a startup because that money is temporary and it's going to dry up. So we're going to have a couple of cool pilots and demos and then at some point it doesn't work and we'll move on to the next hype train. The kind of collaborations that you want to start. If you're an AI company, you really want to figure out like, how are you going to get a real ROI for your downstream customer and not how are you going to build a cool gimmick that they can showcase to their CEO and get promoted and then disappear.

Next up is product roadmap. How have you invested in making your product kind of enterprise grade or when you were saying backstage stage, first as a startup you build a product that does a good job, then you have to build twice as much to make it something an enterprise can buy. So I'm curious, what have been the most important investments that ensured that a future like Fortune 50 company can actually adopt lean, anthropic, contextual, et cetera.

The first thing, just working on security aspects and compliance. So getting your SOC2 certification, compliance, GDP, FedRAMP, as Ben mentioned. So that's one stream of enterprise requirements, which is you just need to work on all these compliance matters. Then in terms of the product, depending on what your product is, there are going to be like a lot of demands that enterprises are going to place on you. For us, we actually ask enterprises to actually give all of their data to us so that we can make it searchable for them. And so now imagine a large company like Nike talking to a small startup like us, which is in stealth and saying that, oh yeah, I'll ship all my data to you. That's just not feasible. And that's not an acceptable answer. So you have to figure out like how you can actually work through that. One of some of the things that we had to do or basically support running Glean within the customer's cloud environments so that they can have the peace of mind that no data has to leave their premises. That was one of the things. And then I think like ultimately as you start to gain some momentum, as you acquire some customers, then, you know, things become easier for you because now you have reference customers, you have actually other customers have placed trust in you. So get some trust by transactivity and yeah, so some of the things that.

We had to work on, yeah, we talked about the Amazon and Google partnerships and a lot of companies are already on those clouds and being able to not have to go through procurement because any service just works for them is a big thing.

Yeah, so I think there's an obvious answer around like SOC2 compliance and all that kind of stuff, or working with partners who can do all of that for you, because as a company often you're not fully certified for everything quite yet. But I think for me personally and for contextual AI, how we're approaching the problem is going back to the drawing board essentially. So we're saying language models are great. Ideas like vector databases are great. All of this stuff is definitely very promising technology, but because it's not quite ready yet, we need to fix all of these problems still. So we need to fix hallucination and citation and attribution and all of these different things. And one way to fix them is to go back to the drawing board and think about what does the language model really look like if you want to satisfy these enterprise requirements. So if you want to have something that doesn't hallucinate and that has attribution, what does that look like? And so we think that's not what we now call a language model. So it's not one big parametric monster. It's something a bit more elegant that has the retrieval kind of built in. So retrieval augmented language model and is trained end to end so that it can rely on information that's external. And so that means that it will hallucinate less, it will have attribution, it will stay up to date, it will not have compliance issues. You will have a very clean separation between the data and the model plane. And so I think making language models truly enterprise grade might require having a different kind of language model in the first place.

I wanted to bring up maybe a couple subtopics we didn't touch on, which is fine tuning and training data. I know you're probably getting a lot of questions about fine tuning and model training from people who have never done it in their life. And I'm curious, what are the concerns and expectations people have from this? Because I think maybe this is also more relevant in the diffusion world. But there are a lot of copyright concerns over training data. And even with LLMs, like there are companies being sued about having, you know, used copyrighted data in the training models. And I know that there's some hesitancy from enterprise buyers on, even if we didn't do this, is there a liability for us if we use these, if we use the output of those models? And does fine tuning like protect us? Is it enough to give us a layer between what the original like model could do and what we would do for the enterprise? So I'm curious, like how does that topic come up for you and what are people wanting to do with fine tuning or having custom training data for models?

Yeah, fine tuning comes up all the time from people who don't really know or understand like how these language models work. But there's so much, I think when the media covers AI so much like everybody's coming in with a lot of assumptions or like things that they think they need. So it's a lot we have to like we end up actually doing a lot of sort of training and level setting with our customers. On let's not talk about individual pieces of technology like fine tuning or training models. Let's actually focus on back to business use case, like what are you trying to actually solve? And then think about yeah, maybe it will require you to fine tune models, maybe it won't. And there's the alternate approach which is what you were saying, which is use knowledge engines as your retrieval layer and combine them with the power of LLMs. And so that's our approach as well at Glean. And it's just, I think you just have to spend time with customers and work on their use cases. And for us it's easy because we are like very much internal focused.

Right.

So therefore those other concerns you mentioned around am I using copyrighted information and all that hasn't really come up that much for us.

Does it come up for anthropic?

Oh yeah, definitely. One thing that recently happened, I think Maybe yesterday is GitHub said that they would take liability for all copyright suits against GitHub copilot usage. That's a really interesting legal precedent. We'll see what happens there. In our case, we do the best we can with the data where we respect things like robots txt on websites when we scrape the web. So we try to do our best to make sure we're not taking data that people haven't said okay is okay to take. There are also websites that let people register to say don't train on my data. And we respect those. In terms of the fine tuning concerns, I totally agree with Arvind that a lot of people don't understand what they want out of fine tuning.

Yeah, we see this a lot actually where customers just go, we want to fine tune our model. Can you help us with this? And so what we tell them is you've probably been lied to. You don't have to fine tune your model. It feels very intuitive. We have this great data set, we own it, it's our data. So we need to do something useful with it. So we need to fine tune our own language model. And so the companies who are offering that service, they would like you to think that. But I just think that's plain wrong. So you can just solve that problem through retrieval, augmented generation, or you can do it by having a very long context window. You really shouldn't need it. The only case where you can need it is if you want it to support a use case where you have a lot of data that nobody else has and it's really specific to that use case so it's not continued pre training, supervised fine tuning sort of things. It's really like I want to specialize my model. I don't want artificial general intelligence. I want to add artificial specialized intelligence. And if you want to do that, then you can use fine tuning. But that's a very different approach than what people usually think when you say fine tuning.

Awesome. All right, well, we'll do a very quick round of predictions. Who wants to kick us off for a great 2030 prediction? I would love to learn what is something wild that you hope will be true in 2030 and then also what's like the more realistic likely thing that will be true.

I don't know about something really wild, but I think what a more practical hope. And I think it'll happen. I think all of us will have a truly smart, knowledgeable personal assistant that we can that will always be our companion and do like most of our work for us. I think today that luxury is limited to executives in the enterprises. And I think in the future it will be true for all of us. That's my hope.

I guess my big bright future is that language models understand us better than we understand ourselves. And when we ask it to do things for us, it will not just do what we say, but do what we meant or meant. If we were 10 times smarter and had 10 years to think about it, that's what it would do. And it would. It would make us all better people and improve our relationships and help us be the best versions of ourselves. That's my. My big bright future. And then for what it will actually be, I'm pushing for it to get as close to that as we can. Maybe we'll fall a little short. I know KRS they say 60% is success, so I take that.

Yeah, I think the technology has a lot of potential to do good. And I think it's really going to be a very different place. And so what I think is going to happen is that there's a couple of CEOs on the stage here and I'm sure there's a couple of CEOs in the audience. So we will all be our own CEO of our little company of AI co pilots that are going to be doing a lot of very boring, mundane stuff for us that we don't want to do, which is going to free up a lot of our creativity and a lot of time for us to do things that we really enjoy doing. So I think, yeah, we have a bright future ahead of us.

Thank you, thank you. Thank you so much for coming.