“The Why, How and When of Generative AI for Startups” with Google Cloud

Foreign. Ravi rajamani, managing director and global head of gen AI at google lillian breidenbach, co founder and managing director at legalos. Harpinder singh, partner at innovation endeavors and sarah al husseini, co founder and coo at ultimate.

Good afternoon everyone and I'm thrilled to be at sasa. Today we have an awesome panel. We're going to talk about trends opportunities with Generative AI. Just before I introduce the panel, a little bit about myself. I'm Managing Director, I lead the global AI Gen AI Black Belt team. So this is a customer engineering team. As far as our panelists go, we have an awesome panel of two of our staff startups that are part of our Google for Startups cloud program and our VC partner from Innovation Endeavors. I would like to introduce the panel to have an opportunity to introduce yourself. So let's start with Lillian.

Thank you. Hi everyone, I'm Lily. I'm the CEO and co founder of LegalOS. I'm actually an anthropologist by training but I've been working at the intersection of law and tech for the past four years. LegalOS that we're building an AI powered legal assistant for revenue driving functions. So offering legal support on demand for those teams in Slack and Ms. Teams or in Salesforce. So for example answering privacy questions, drafting emails in response to pushback on the SLA or vendor forms, et cetera. We're Berlin based and we work with scale ups and enterprise all across Europe and the us.

Hi everyone, I hope you can hear me. I am Sarah. That's me. I am the co founder and the COO of Ultimate and we've been building ultimate for about six years now and we're about 170 people across Berlin, Helsinki and London. And ultimate is a customer service automation platform powered by AI and Generative AI specifically and brands from Footlocker to Finnair and Zalando to Zendesk. They use us to build a bot that works as a core member of their customer support team and automates customer requests across all of their digital support channels. That's messaging, email, chat and at native fluency in every language.

Awesome. Hey everyone, good to meet you. I'm Harpy, I'm with Innovation Endeavors. We are in early stage VC fund. It was actually started by Eric Schmidt when he was a Google CEO and one of my other partners Drawer and we primarily invest in seed and series A companies in two areas. One is AI applied to big physical world problems. So kind of problems that you two are solving and also enterprise infrastructure. So machine learning, tooling data, infra dev tools Areas like that. My background is mostly as an engineer and a founder. I was a founder for about 18 years. I started two companies, one in wireless infrastructure and second one in data analytics. And I led product and then I was a CEO at my companies. And in addition to being at ie, I also teach at Stanford. I teach a couple of entrepreneurship classes at Stanford. Gsd.

Thank you. Thank you, Harpy. Thank you, Lillian and Sarah. So Lillian, I would like to start with you first. As a startup, one of the more complex things is as you try to integrate generative AI into your product roadmap with the fast evolution of the technology itself. What has been your experience and can you share some of what has worked and some tips for other startups and co founders?

Yeah, sure. So I think for us it has, it was a very radical process. We've been trying to solve this core problem of teams in an organization waiting on legal for the past three years. And we first started with a programmatic approach and it only really just got us that far. And then almost overnight with the emergence of LLMs, this became a solvable problem. So at the end of last year, we actually scrapped our entire roadmap and I would say almost 90% of our code base and started building a product almost from scratch. And it was like a very painful experience, but 100% the right way for us to go because we were now able, or we are now able to solve this problem that was unsolvable before. And it also gave us the freedom to actually build a product. Every design decision, every product decision we made was informed and powered by this new technology. And it just made a much more easily adoptable product that actually solved the customer pain point. And I think the two pieces of advice I would give people is if this is going to make you solving the customer pain point like that, much easier, that much better, then you need to go all in and you gotta kill your darlings a little bit if you have the competency in the team and if you also. We were early stage enough to be able to do that. And the second I would say is like allow people to prototype, for example. We exposed all of our prompts to the entire organization, not just the engineering team, and it enabled us to iterate so much quicker. Because the beauty of startups is that you're agile and with LLMs, it's language driven. So it's not limited just to engineers. Yes, some of it is, but that was a complete game changer for our entire organization.

Yeah, I mean, as you said, generative AI, it seems to be a very transformative technology. And it's fascinating that you were able to declare bankruptcy and you know, go get rid of the technical debt. What about you with Ultimate? Like, how was what, what has been your experience, Sarah?

Yeah, I mean, firstly, I think that's a really great question that you ask yourself at the start. Is generative AI going to be transformative for your product?

Right.

Or is it going to be incremental? And if it's going to be incremental, it's a really low risk thing that you're working with with your product roadmap. You know, you can test a lot of different technologies, you can decide to use something open source because it gets you live fast and you probably don't care that GPT4 is expensive and slow because your usage is going to be low because it's a little incremental addition if it's going to be transformative like it was for Lilly, like it was for Ultimate. So we are like I mentioned, a customer service automation platform. So conversational AI is like the core, core, core technology. And so generative AI was just like an existential discussion for us. When it's going to be transformative, you face conversations at the start like, you know, do you want control over the technology? Do you want to be building your own LLM and building it all in house? Or if you're a startup, which we both are, is time to market absolutely critical for you? In which case are you going to be using open source, wrestling with something open source while at the same time building a plan of action for taking it in house long term? And I mean ultimate, we've spent the last six months, you know, wrestling with this and essentially trying to put the genie back in the bottle. And it hasn't just been one thing that we've built, you know, it's hundreds of different optimizations, AI features, different tweaks in the whole AI pipeline to get this working. Because the truth is, when you're using open source and you've seen it yourselves, it's very, very easy to build for a developer to build a really cool demo, right, which is about 80% of the way there. But to get from that 80, 90% to 100% something production grade. And in our case that means an AI virtual agent that is honest, that stays in scope, that is controllable, that's, that's the Herculean efforts. And so I would say my advice would be to assess the risk profile of your customer base, right? Are your customers going to be okay with a Better product that has these kind of, that can go off piste while you work out the kinks or does it need to be production grade from day one, in which case you need to be taking on much bigger challenges and maybe bringing this tech in house. But I agree with Lilly that if it's going to be transformative for your product, for your industry, then there's no time like the present. You need to be starting now and you should probably be thinking very ambitiously about how much resource you dedicate to this.

Yeah, thank you Sarah. I'm going to come back to you both because you, both of you mentioned transformative. You know, we should talk about opportunities. But before that, Harvey, from a venture capital perspective, you know, as startups come to you or as you look at startup startups and their roadmaps, what are you seeing or what do you recommend as advice to other startups?

I think two or three things. One is Genai is super fast moving obviously so the basic things around the right team in the right market with a solution that is 10x better, I think that's still equally as relevant as it was in the last 30, 50 years of building startups. I think second is like you two talked about, I think you deeply thought about the customer experience and what generative AI is going to do for you. I think we see a number of startups which are building this thin layer on top of OpenAI or whatever stack. So I'd be thinking about the durability and defensibility of your value prop, especially as these general purpose foundation models get better. If you look at some of these copywriting companies out there or maybe even creative and image generation type of companies out there, the general purpose model is just improving and I think in two, three years I would be worried about them eroding the value prop. So, so I think we like founders who I think like Lily, you said are not just sprinkling AI on top of the product. They've basically fundamentally rethought, you know what Gen AI means, thrown away 90% of your code and have deep customer empathy for the problem space that you're attacking. I think the deep customer empathy meeting sort of a real transfer transformational use case of gen AI. I think that's, that's what creates sort of the magic here.

Amazing. You touched on one thing and the 90% from a developer standpoint and I'm kind of going off script. So it's, it's important for us to talk about what is the role of a new age developer. Right. So the Technology is so new. You know, some of us who are used to the traditional side of software development, even you know, agile, all these new things, do you see that as companies as a strength versus an enterprise that might not have the flexibility or the velocity to move fast? Do you look for that in a startup?

Sorry, just to rephrase the question, are you looking for developers that have the.

Versatility or how should we be looking for recruiting and how should the engineering organizations be set up for, for taking advantage of what the speed of LLMs are like? Generative AI?

Yeah, this is all about iteration, speed. I think the beauty or the curse of this market is the same toolkit is available to everybody. Right, right. And it's really how many experiments can you run in a finite amount of time? How much can you incorporate your customer feedback back into improving your models and improve the UX over time? So it's all about speed. So I think teams that are nimble are able to out execute everybody are going to win.

Got it.

With a lot of these business models there is, it's all about product and ux. There is no sort of deep proprietary moat, especially for some of the SaaS businesses.

And as you look at it from a challenge and sort of opportunities and challenges in this field. Sarah, how are you looking at it from an ultimate standpoint with generative AI?

Yeah, I mean, I mean it's widely agreed that customer service is one of the number one industries that's going to be disrupted by AI, especially with generative AI, with how human like the experience can be. And so ultimate really sits at the heart of that. And we get to, we get to witness it. And I mean generative AI, once it's evolved a bit further, because the industry is quite rudimentary and the applications are. But once it's evolved a bit further, it will be used by companies like ultimate to make it incredibly easy for support teams to deploy bots to that work across channels, that work across languages, that work across systems and that automate probably 80% of customer requests in a really human like way. So for us in the next 12 to 18 months, it's going to make chat bots launch so much faster. It's going to make them much less effort to manage. You're not going to need to train or think about intent hierarchies ever again. It's just going to go live.

Right.

Which is just, yeah, awful for users right now. And finally, of course the experience is going to be so much more human. Like it's going to be very hard to tell if you're speaking to a human or a machine.

Yeah, yeah, fascinating. Lillian, same question, but from, from the industry that you are servicing towards more, you know, the legal regulatory environment where customers are more risk averse, they are worried about, you know, data privacy and, you know, other sort of considerations. How are you looking at it from AI helping you from an opportunity or, you know, what are some of the challenges you see?

Yeah, so 100%, right. We're dealing our end users, our sales teams, our marketing teams, but the people who buy us and set us up and administrate the product are legal and compliance team. And if you give them an AI product that's a black box, they're not going to use it. Right. Because they can't. Because if these bots are going to give advice on how to negotiate with a customer, then it has to be accurate and it has to be secure. So from the very beginning we were like, okay, well, we need to focus on making this AI explainable, steerable, auditable, and really put our focus into that. And that came with its challenges and also with its sacrifices. I would say, like, for example, we definitely sacrifice speed of response. Our request takes about one to three minutes. Now on the grand scheme of things, teams are used to waiting for four hours, seven days, 30 days in certain enterprises. So two minutes is nothing. But it's slower than maybe you'd expect at first. But it's a sacrifice that everybody's willing to make. So I think that's definitely something that we looked into. And, but I think the beauty of it is that if you can build steerable, auditable, very secure AI, then these teams develop a huge amount of trust into the product and will deploy it very quick and very widely. And I, you know, I've been in like legal tech for four years. I've never seen technology be deployed that fast or at that kind of scale ever before. And that's so cool to see.

It's, it's interesting you mentioned innovation, but more responsible innovation. I think that's, that's the key, especially in yours. You are working across multiple industry verticals. If I may say, Harvey, are you seeing any patterns or what sort of trends are you seeing with respect to gentle in each of these?

Yeah, so I think, couple of things. On responsible innovation, I would just say the only advantage that startups have is they can move fast.

That's right.

So you can take risks that Google can't, that Procter and Gavel cannot. So I think it's okay to push the envelope. I Think the other, other couple of things we are seeing, I think marketing vaporware is powerful in this space.

Right.

All the big companies and small companies are claiming everything. So I think as a startup it's really difficult to differentiate yourself. So I think maniacal customer focus, getting the right case studies and so on, sort of really important to kind of break out from that noise level. And third, we also see, I think the bigger companies, I think in previous generations of innovation, even if you look at AI and machine learning as it was deployed, I think the bigger companies weren't very well suited to do that. General Motors couldn't attract the same talent that maybe Palantir put here, so they weren't able to move as fast. But I think the toolkit here is sort of so democratized that anybody can build a POC and prototype and put something up within like 30 days, 90 days. It takes a while to get better at it. I totally agree with the comments here. At startups we've seen, it takes three to six months to find you the models and get them to actually work, but very easy to claim that you've got all this stuff. So I think as a startup you got to be nimble on the product, but you also got to be very, very savvy on the go to market.

Yeah, this is. Thank you. I think I want to be respectful of the time. So can we take maybe 45 seconds each and sort of provide your closing thoughts on. On how you are looking at this from. And what would you advise your startup communities? Just go from Lillian all the way to Harpy. Sorry, can you repeat that just to be a closing comment?

Okay, sorry.

Addressing the audience.

Yeah, no, I mean, I think for like, like I said, I think Genai for us was completely transformative. Transformative for the entire product, who we sell to our market, the problems we're able to solve. And I think it's so exciting to be working kind of at the cutting edge of technology right now and actually defining the patterns also. Right. Because this is such an untapped opportunity. No one knows what it's going to do to the industry, no one knows what it's going to do to legal, and it's exciting to be on that journey.

Thank you.

Yeah, I completely agree. I think it's an absolute gift. I have had so many great years building Ultima and I would say that this is the most exciting year we've had so far. And it's really down to the privilege of being able to play around with such a new technology. I completely agree with how penda that it's all about speed for us. We recognize that immediately. It was all about launching something fast, getting something better out and listening to the customer feedback and building an organizational system internally where you can capture that feedback and it's right upon it. Incredibly quickly we scrapped our product roadmap at the start of this year. We stopped working with product entirely and we only worked with a Sprint cycle. We scrapped full year goals and only worked with monthly goals. We've just readopted quarterly goals because we're able to see a little bit further ahead. That is the pace that you need to be moving out to be able to keep up with what's going on.

The I think in addition to speed, I think the only other thing I'd bring up is if you look at some of the previous waves like cloud and data and so on, I think we're like the 15th year of cloud here, 18th year of cloud probably and it's still not done. So I think this is going to take a while. So I think pick good long term partners, I think speed is super important. But also be thinking about go to market your Act 2, Act 3 and so on. I think there's going to be some very large companies created over the next sort of 20 years here.

Three of you, thank you very much for your time. This was a really interesting discussion. It feels like a tectonic shift in technology here and thank you all for being here. I know this is a busy day. The energy is palpable in the floor. You know congrats on all the innovation that you are driving just a little bit. We'll be outside. I know we don't have time for Q and A, but we would be on our Google Cloud booth in these 505 is if you want to stop by, ask us any questions and good luck with your innovation and whatever you're learning to do with AI. So thank you for being here.