Tomasz Tunguz Theory Ventures on "AI: The Where, The When, The How" with MotherDuck, Lamini, OpenAI

Ladies and gentlemen, please take your seats. Our program is about to begin. As a courtesy to our speakers and fellow attendees, we ask that you please silence your cell phones. Please welcome to the stage Jordan Tagani, founder and CEO at Motherduck. Sharon Zhao, co-founder and CEO at Luminai. Maggie Hott, GTM leadership at OpenAI. And Tomas Tunguz, general partner at Theory Ventures.

Hi everyone, my name is Tomas Tunguz. I'm founder and general partner at Theory Ventures. Thrilled to be here with you all. And our illustrious panel of guests. Today we're talking about how data and AI are changing the way that applications are built. We've got an incredible group of people here, starting with Jordan Tigani and Sharon Zhao and Maggie, and would love to just kick things off by having them introduce themselves and we'll jump right into the agenda. Jordan, do you want to take it away?

Yeah, sure. I'm co-founder and CEO at, at Motherduck. We're building analytics engine based on open-source DuckDB. I helped create Google BigQuery, led them for— led that for almost 10— or as part of it for almost 10 years, uh, before, uh, before starting, starting Motherduck about a year ago.

Hi everyone, I'm Sharon. Uh, I'm the co-founder and CEO of Llama. Uh, we're making it easy for enterprise customers to be able to fine-tune their own LLMs. Most recently, I was computer science faculty at Stanford teaching generative AI, where I also got my PhD with Andrew Ng. I teach about a quarter million professionals online generative models, fine-tuning LLMs most recently. I'm very excited to be here today.

Hi everyone, my name is Maggie Hott. I'm on the leadership team at OpenAI, in particular overseeing the team that is bringing ChatGPT Enterprise to market. We launched about 10 days ago. Prior to this, I spent 2 years at Webflow building out the sales there. And then prior to that, 6.5 years at Slack, from the very first founding sales hire out of headquarters all the way through acquisition to Salesforce.

Incredible backgrounds. Well, I'm really excited to talk to you all today about the way that data and AI are changing the way that applications are built. It's just been an incredible last 6 months. Look at the rapid pace of innovation within LLMs, how they're really transforming a lot of the different ways that people are using software. And maybe Jordan, we can start with you. At the data layer, you built— you're commercializing DuckDB, one of the most popular databases now in the world, super high performance in-memory database. How are you seeing AI change the way that applications are built and the way that people are using data?

So to start out with, I was a little bit of like a resistor thinking, well, you know, this isn't really going to change our world. You know, maybe it's changing a lot of other things, but Then I was talking to somebody who was, he was giving a course on data thinking at Princeton, and he was, you know, teaching people how to ingest data, how to query their data, and then how to visualize their data. But he wasn't teaching them any programming languages. He wasn't teaching them SQL, and he wasn't using like higher-level tools that can automatically kind of do this stuff for you. He was just using ChatGPT to generate Python code to ingest the data, to generate SQL statements, to generate like the— then the Python, you know, visualization, visualization stuff. And I'm like, wow, like, you know, and this was like— this was a few months ago when ChatGPT had just been out. And I just— you realize that like, yeah, this is going to change who, who can, who can do, who can do this stuff, um, and, and also change the skills that people are going to need and, and the tools that you're going to need to build So I think that's one side of things. And the other side of things is just the developers, people who are building things with, you know, that need access to data are going to be changing how they access that data. I think the V1 is like the vector databases. But sort of I don't even think we know necessarily what V2 is going to look like because, okay, you get those vectors out and, okay, what do you do with them? How do you actually access, um, access your data stores and relate that to, um, relate that to what you're getting out of the models. So I think there's a whole big area of, uh, of unknowns and exciting possibilities there.

Yeah, I mean, one of the things you touched on there that I think is really important is how much LLMs are democratizing technology. It's really much easier for someone who's maybe not be as sophisticated with SQL to suddenly start asking and answering questions of their data. In ways that would have taken them 3, 6, 12 months to learn beforehand. It's, it's an incredible phenomenon, and it's being adopted broadly within the enterprise, right, Sharon? I mean, you're seeing this, we're seeing this just kind of across the board. You look at the total number of customers that are starting to use Microsoft's OpenAI technology has 3x, I think, in the last quarter in the reporting. How are you seeing enterprises start to adopt some of these technologies, and how are they manage it? There's a lot of technology here. It can be complex. There's, there's tuning, there's lots of different components to it. What are you seeing in the market at Laminate?

Yeah, I think even just zooming back out at a technical level, I think actually public data is running out for all, all of what LLMs can take advantage of. So actually the next frontier for LLMs is in enterprises, and I believe the best LLMs for this next, next wave essentially will be enterprise LLMs. And the enterprises that I'm seeing that are, you know, at the forefront of this, they have started to adopt techniques around fine-tuning their models to personalize models for every single user and every single customer of theirs where the data doesn't actually leak between those customers, but the model is highly personalized according to their access controls for data, and just any kind of complex— any kind of complexity around their data slices and use cases. So that's what I'm seeing, and it's honestly really exciting to see this because this will further democratize all the possible use cases that can be out there, you know, far beyond what I can, I know, even imagine.

Yeah.

So, okay, so what we're starting to see— we're meeting lots— I'm a venture capitalist, we meet lots and lots of different businesses. They're starting to offer these software that's enabled with AI. And there's a question that we're sort of wrestling with in the ecosystem, which is who— like, is it the vendor who's responsible for actually optimizing, or are there people inside of the business that are subject matter experts? Are you seeing any trends on which side is going to be responsible or mostly responsible for optimization?

Yeah, so there's definitely this tension between the two where AI expertise, You know, it might take a couple hundred top AI researchers to be able to train some of these models such as OpenAI. However, I believe that, and this is based on my experience training these models, it's actually the domain experts will be driving the best models out there. It won't be people like me who can actually do all the model training, etc. All that infrastructure will be built out such that more people can actually define the trajectory of these models. And as we saw with ChatGPT, with more people being able to use these models, they thought of way more creative, creative different use cases than like I in a silo with a few other AI researchers can think of. So I think the trend we're seeing is, uh, really, you know, the, the typical developer, the infrastructure developer, even the front-end developer, all these developers are able to— who are able to access that data and be able to manipulate that data are the ones who can actually help develop the trajectory of these models and do the fine-tuning and actually build out this next generation of LLMs for the enterprise.

Yeah, that's a great segue. Maggie, you're in the center of the world at OpenAI, basically catalyzed a big chunk of this movement. You recently launched OpenAI for Enterprise. Can you tell us what was that like? What, what are you seeing in the market? What's What sort of revelations can you share?

Yeah, absolutely. So I will actually break this down into 3 things. I'll share with you all a bit about our journey to building ChatGPT Enterprise over the last few months. I'll also share with you some of the learnings and kind of use cases and what we learned from working with our design partners. And then finally, I'll share a bit about what is coming, which we're really excited about. So I was hired on to OpenAI in April, really with the mission of building out the go-to-market team for ChatGPT Enterprise. We knew it was something that we were going to build, and that was because when ChatGPT launched early last December, all of a sudden we started seeing millions and millions of knowledge workers using this tool every single day for their jobs. But then you have kind of the, the age-old security, privacy, IT side of the house that is saying, whoa, we don't want our corporate data being put into these models and trained on these models. So we put our heads together and said, let's build an enterprise version of this that has things like data training turned off, SSO, SCIM— I mean, all the enterprisey features— SOC 2, CCPA, GDPR, all the acronyms. So when we set out to really launch this, what we decided to do was to select a pretty big variety of design partners. So for example, Amgen, PwC, Bain, Samsara. We decided to go as wide as we could with different industries, uh, and use cases quite frankly, because we wanted to see how healthcare uses this tool. How does banking use this tool? How does consulting use this tool? Uh, so we worked really closely with them behind the scenes, of course, you know, under some pretty mega NDA lockdowns for the last few months to understand what is it that the market wants to to see for a V1 launch. Throughout this, there was a really big learning that came out to us, and it was the importance of developing alongside your customers the idea behind really strong use cases enablement training. I think when we started off, we just kind of assumed, oh, everybody knows how to use ChatGPT. But then we realized everybody uses it in a different way. That they've kind of created themselves. So we've been thinking very deeply about how do we work with these design partners and now the world to develop very specific tailored use cases to their industries, to their personas. ChatGPT can be used by every single vertical, every single company, every single department within a company. It's just mind-blowing. So that was a really big learning for us, was really invest in customer customer success type work early on. We set up Slack Connect channels, we asked these design partners to have weekly syncs, we sent out surveys to all of their beta users, and so we feel really good about what we launched last week. And we've also been quite public with our roadmap and what we're building here, which I think is also quite unique. Normally companies are a bit cagey, and we've shared pretty broadly that the single biggest thing that we are working on right now is this idea of customization. So the— imagine being able to have your entire company's corpus of data, or all of your own personal data, infused in with ChatGPT Enterprise, and basically being able to query, you know, all of my top sales deals, what were common trends across them, and having that all come back for you. Um, the other thing that we are also building right now, which is really exciting, is a business version. So more of a self- a faster version for SMBs. We realized, you know, we went out of the gates with enterprise, but a lot of times SMBs just don't quite need all the enterprisey features that we launched. Uh, so there's just been pretty huge demand from SMBs, from startups, from founders to be able to adopt this. The use cases are endless. Uh, Code Interpreter, which is now renamed Advanced Data Analytics, is probably the most powerful tool I've ever seen in my life. You can essentially ingest in hundreds of thousands of rows of data or spreadsheets. I think about it as a super calculator. I use it personally for looking at leads, looking at sales data analysis, opportunities, win-loss rates, everything. And it basically just synthesizes anything that would typically take someone a day to parse through in minutes. And then down to the bare bones of recapping customer calls, feeding in transcripts from Gong and having it analyze that and then write a Next steps follow-up email that is like beautifully perfect, personalized, and tailored, which would have taken me half hour per email before, I can now do in 10 seconds.

That's amazing. I think the point that you hit there, that solution sale where you have to take something that's a general purpose, real innovation, and really solve an end customer problem, that's what we're seeing in the ecosystem today. It's like people were really excited about the technologies and industry, but we need to figure out, okay, what is the problem that the end user really needs to solve? Sharon, Jordan, are you starting to see, like, are you focused on individual personas when you think about taking your products to market or seeing different use cases arise?

I think, you know, there's the sort of developer persona, and I think those, you know, I think the way those, I mean, as you mentioned, that everybody uses ChatGPT in a different way. So, the developer is going to use it in a different way than an analyst is going to use it, and which is different than a data scientist. And I think, yeah, there's going to have to be different tooling for each one of those.

We're hyper-focused on the developer, so the software engineer, someone who can actually deploy a real app in production. And we're just really focused on getting these basically production use cases out. And we believe that the, the software engineer who sits most closely to their production apps will be the one to do that. And we've been finding that they're able to leverage data really effectively and efficiently. They're able to hook up their internal systems, whether that, whether that be recommendation systems or search systems, into these models as well. So they're, they're very capable at pulling together all of these different APIs and continually improving these models on a schedule. And so we're hyper-focused on this persona, but I'm also really excited to see just so many people be able to use this technology.

Yeah, that's been a big change. I mean, 10 years ago The way that I thought about the data stack was you had systems that produce data, and then they would put it into cloud data warehouses or data lakes, and then there were 3 consumers. There was the BI use case, exploratory data use case, and then there was machine learning. You're trying to build data products, and now it really seems like those machine learning engineers or that functionality is becoming part of the core products, becoming part of the core workflow. Like, Mag, you talked about Gong analyzing those sales calls. That's part of the product. The machine learning engineers are now starting to be put into the path of production. There's this blend, there's this mix between— okay, you've got a classic software engineer who's used to shipping code with CI/CD and the classic workflow, and then you have machine learning experts and bringing in different technologies, whether it's data caching or customized models or off-the-shelf APIs, and you're putting it into production. It seems like there's a merger there of two personas that historically have been separate. And the landscape is sort of rising up to how do we enable those people to be as effective as they can be using things that maybe their expertise is sort of out of their domain. So you need a higher level of abstraction. Is that what you're seeing in market, Maggie?

Yeah, absolutely.

Yeah.

And I think that's incredibly important actually merging those two together because historically, maybe to give a fun anecdote, one of my friends used to lead engineering at Uber and so a 1,000-person engineering team. Team, and what he found was the machine learning team, you know, was building models in Python notebooks that were obviously then had to be sent to the production team to then, um, you know, like ship it into your— into all of our apps. Uh, and one time, because Uber was, you know, moving very fast, let's put it that way, uh, they had like just fired the person who ran this other model, so they had to literally go find the person's personal laptop open up the Python notebook, rerun the data on these like a few states where they had to like have an updated model, and then somehow rip that model out. And thank God they found that laptop and put it into production, and now it's shipped on our phones. And that is an insane process that's super brittle. Um, and I guess another anecdote is just, uh, you know, it's really, it's really hard to get some of these systems up. And in machine learning, you know, a lot of AI people are like, yeah, we push the performance of this model, and we define performance as accuracy, or, you know, accuracy along these like general benchmarks. Maybe it's 6th grade science questions or something. But a production software team will look at that and say, that's not performance to me. I also care about the speed, the latency of, you know, this API call. You can't give me a better model and give me 100x slowdown of this model because I'm putting it in an autocomplete type of interface. That's not going to work. And so I think merging these teams together, bringing them together is really important to actually shipping these models in production. And I think historically it's largely been this like R&D team with a really thin line to production and oftentimes failing that. And only, only very few teams could succeed at that before. But I think now many more teams will be able to succeed at that. And it's just really exciting for, for AI more broadly.

Yeah, and I think as those, as those teams are merging, there's, you know, there's this sort of interesting debate which is what is the lingua franca of, you know, what, what language are people going to be writing in? Is it going to, you know, like, there's— is it Python or, or is it, is it English? And so as, as people start to be able to use just English natural language to do some of the tasks that were in the other, other camp, I think that also can help accelerate that, that merger, even if sort of maybe fully getting towards, you know, you describe your program as sort of like an English language paragraph and have that turn into a production system. Maybe that's a little bit further, further off, but I think as you start to kind of peel off workloads, it can just help those teams merge.

That's the thing that's so exciting. I mean, think about Microsoft Office, 330 million paying subs, whatever. Let's say a third of them use Excel. All those people want to be able to access some more sophisticated data within the organization. They want to analyze it. Like you said, historically they've really needed to learn either VBA or SQL. It's a language not everybody really knows, but now if they can start to speak English, you really have the power of data now being exposed to, I don't know, 10x, 100x greater the population. It's pretty incredible. So just as we've seen, I mean, what we're talking about on the technical side of an organization, you really see a big transformation in what's happening. Maggie, you're a sales leader. How are you seeing LLMs and these applications change the way that companies are starting to go to market? What's going on in the front of the house?

Yeah, I think it's best to start with a bit about how sales has changed over the last few years, because every year it seems to be there's a new step. And I think the, the short of it is that AI is going to fundamentally change how the sales role operates and looks, and it already has. We're already well into that. 2017, 2018, 2017, it was pretty easy to prospect into companies. You know, prospecting is kind of the bread and butter of any sales organization to generate opportunities and customers. Uh, then all of a sudden you have these tools of Salesloft and Outreach, which come out and just take prospecting mainstream. So then all of a sudden sales reps are sitting there saying, what do I need to do next to get in front of buyers? The next step was personalization. So years ago, personalization wasn't even a thing. Last few years it's been all about how do I research this company, this customer, this person? Did we go to the same school? What can we kind of connect on together? Well, now LLMs have made it so research and personalization can be done in less than a minute. In less than a minute you can have ChatGPT or Jasper or any of these other tools that are out there right now go write an email that sounds very personalized and tailored but was actually written by an LLM. So now this kind of next era of sales and sales orgs has to really be thinking deeply about how to interact with the customer. No longer can you get on a call and ask somebody, you know, a million discovery questions. You have to show up to that call with a strong point of view with how your, your product is going to save this company money or solve their pains or whatever their mission is. And the beauty about LLMs is now they can help you do that research in advance. So back in the day when I was a sales rep, I used to spend 2 to 3 hours before a big meeting, even more, researching and prepping and practicing my discovery questions and coming up with a value hypothesis. Now that time can be significantly reduced to maybe 20 minutes max, and you can literally have the LLM be role-playing with you of the questions that you are going to ask and the value statements that you're gonna bring. And now I have, you know, LMs also on the other side of it are going to save so much time for the sales professional in terms of administrative tasks. No longer are the days of writing, as I mentioned, long follow-up emails or logging your notes into Salesforce. You can literally feed, as we talked about, transcriptions from Gong directly into there and use prompt engineering engineering to write these beautiful summaries. There's all the new tools that are building slide decks for you, which is wild. So you're able to save on the administrative side of things, but the sales rep in this day and age and sales teams have to really step up with their value selling.

Do you think that changes the structure of a sales org at all, or do you think we stay with like the SDR, AE model?

I think we stay with it for now.

Yeah.

And then I think AI, I think, sales teams and companies that adopt AI are going to significantly outpace those ones that don't. So I actually think now, like, the common sales rep should be doing everything they can to actually learn prompt engineering, to learn how to leverage AI in their day to day to make them so much more efficient. I think maybe someday the model changes, but I don't think we're at that day just yet.

So we're all salespeople, right? If you're head of sales or you're a salesperson, you're a venture capitalist, we're all selling something. I think these tools, like you said, they open up like enrichment, discovery. How are you using LLMs in your day-to-day in order to improve your life? Are there tools that sort of stand out for you where you've seen a complete game changer? Anything kind of come to mind?

Oh, you're asking me?

Yeah.

I think, I mean, I think certainly like people that are interacting with data, it's changing their lives. You know, one of the things I do a lot of is I do a lot of slide decks and like, you know, things like Midjourney and generative AI just make it so much easier to have like the right image. And yeah, maybe sometimes people are, you know, have too many fingers or there's like ducks without wings or like kind of wonky eyes. But I think as the technology gets a little better and perhaps like the operator gets a little better at like prompt engineering, those kinds of quirks will go away.

Well, with Mother Duck, I mean, you've done an incredible job making that such an iconic part of your brand. I could imagine getting the duck anatomically correct is really, really important.

It is important. I think one thing that, you know, we were analyzing our data when it came to all our marketing blog posts or like tweets, everything going out. and there was a 10x increase in anything with an image, right? And so it was just much more engaging for people to actually, you know, have impressions on. And so, um, we found that just spending some time on Midjourney was worth more than spending time on the blog post in any single way. Like, hands down, just like spend a few, a few extra minutes here prompt engineering or, or really just like generating an image of your choice and clicking through them. Versus actually writing out the blog post that no one's going to read. Um, so I, I think like that was a really big, uh, big lift for us when it came to, when it came to marketing specifically.

Yeah, just piling on, I, I, those images are super helpful. I, I was unfortunately sued for using an image where I didn't have the rights, and so I learned a very important lesson. And so now all, most of the images on the, on the blog post that I write are Midjourney generated because now I can be confident that I'm not going to get myself into legal hot water again with rights issues. Yeah.

Go more into that story. I'm kidding.

No, no. Well, so I mean, you know, I was like— you want a hero image, right? Like, you look through the click-through rates and the dwell times. And so I was using Google. And there's a filter on Google where you can search for things that are Creative Commons license. And so I found an image on the Google search results that ostensibly was licensed with Creative Commons. Turns out actually somebody owns the rights to the image, which I was notified later. And so now at least with like a synthetic image, you can be totally confident that effectively, I don't know what the latest legislation is, but nobody owns the image or I at least own the image. So that's been a huge help. Do you use anything daily, Maggie, that really sort of where you see like a pretty meaningful improvement in your day to day?

I would say ChatGPT Enterprise, but that's not a fair one. I have been the last few years, there's just no tool that's better than Gong. Gong as a sales leader, Gong is getting ramped up as a sales rep or a new hire. Gong for product. I mean, as we've been building ChatGPT Enterprise, we would love to have our product leaders on every single call. It's just not realistic. So we can literally take snippets out of Gong and share all the common feature requests. We can drop in hashtags. I mean, I've— I'm on Gong's website on like from my previous company of speaking about them, but there is just no better tool out there than, uh, Gong. So I'd say ChatGPT Enterprise, Slack, and Gong are the 3 tools that I use for many hours every day.

Yeah, that's awesome. We do the same thing when we meet founders. We record the conversation so we don't have to ask them to repeat, and we're starting to save all that information and produce LLMs on top that summarize diligence and We produced our first investment memo summary from an LLM. Wow. And so we can just start to see, like, as we gather more and more of this information, we can accelerate some of the things that are wrote within the org. The other thing that's been really useful for us is blog titles. So, you know, I think about, like, LLMs. I mean, there's these incredible things where they're taking the sum total of, like, human knowledge that we've put on the internet or the datasets that we've trained, and then they can kind of replicate them. And you think about All of the amount of human effort that's gone into optimizing blog titles, that's all on the internet, right? And it's all encoded within these LLMs. And so if I take like a blog post that I've written and then I ask one of these models to produce like 3 or 4 suggested blog post titles, I actually get 3 to 4 times better click-through rates on the titles because I'm aggregating all that knowledge. So that's been a big, big help for me.

Awesome.

Yeah. Okay, so to bring us home, would love like quick take. Where do you see the— for your particular company, like where will we be in 3 years? How will LLMs have actually changed your business?

So I'm going to— a little bit of a cop-out on this one. And it was an answer that a founder colleague of mine gave, which is before ChatGPT, he could sort of look at— look 5 years out into his business and be like, okay, I can see the road and I can see like, I think I know where we're going and I'm pretty sure we're going to be successful. And then ChatGPT comes along and this cloud comes over the road and you just, you can't really see more than 5 feet out in front of you because OpenAI may release something that makes something totally irrelevant or make it so that developers or analysts start to use some sort of other technology. I think the only thing that you know, I think is clear is that people will, people will be doing things differently. Their jobs will be different. I think, for example, like, um, the data space has sort of started to aggregate into these different job titles. Analytics engineer, like somebody who uses dbt, you know, data engineer, somebody who, you know, munges data, analysts, uh, etc. But kind of as all of that stuff can be done with prompts. Like, like, I think that there's going to be another— I think there's going to be a reshuffling of those, of those roles.

Yeah, makes sense.

I think every enterprise will have their own LLMs. And I think what that'll enable is, you know, in every single market, there are domain experts. For example, like sales, they're domain experts. And what an LLM does is it becomes a scalable domain expert. And so for that market, there will be scalable domain experts for every single type of expertise. Tailored to every single type of customer and user. And so I find that really, really exciting. So I think that'll dramatically transform job roles moving forward, and that'll dramatically change how basically how businesses need to operate and who they need to even bring on and hire. Yeah.

Maggie, in 60 seconds or less, predict the future for us.

My— I can't comment on OpenAI's roadmap, but what I can say and what something that I'm the most excited about in terms of an industry that is going to get completely upended is education. Just last week I was with the CIO of ASU, and he has these huge plans to bring AI tutor bots and AI coursework and really democratize the future of education so that anywhere in the world has equal and equitable access to education. And I think that's going to be one of the biggest impacts this world has ever seen, is everything that's going to happen with AI and education. It's it's going to be just absolutely bonkers.

I could not agree with you more. I mean, I think the thing that we've noticed across a bunch of different companies is if you have somebody who's new to a space, new to domain, and that could be math or calculus or biology, it could also be SQL or data engineering, if you can give them access to an LLM, the quick feedback loops and a super patient instructor where you could say like, I don't understand it, or show me this. Or the other day I had written a line of Bash code and I had forgotten what it was. ChatGPT, like, what was it that I actually wrote in this piece of code? And explains it to you. It's an incredible, incredible tool. Yeah, thank you very much. Awesome. So we'll open up for Q&A here for 15 minutes. Any questions from the audience? Hi Maggie, I'd like to personalize my question first by expressing some appreciation for your 20VC podcast. I learned a lot.

Thank you.

Um, as a sales leader, if you're planning next year's sales plan. Do you have any advice on improving an AE ramp time from, let's say, like 6 months to 5 months?

Absolutely. I think you can get— I obviously don't know which, which company or industry, but I think 5 months is actually really long now for ramp time. I think there's a couple different things. Obviously, you know, I mentioned my love of Gong, which is just getting your AEs the previous existing calls in their hands ASAP. But there's also a lot of stuff that you can really encourage them to to do with AI, so, you know, feeding data into an LLM or using ChatGPT and having them create quizzes, basically testing your product knowledge. A lot of stuff you can do with AI, a lot of just kind of traditional type things of certifications and whatnot. And if you ever wanna chat about it, feel free to reach out on LinkedIn, but there's so much that can be done to get AE's ramp time so much faster than it used to be.

Appreciate it.

Yeah.

Hello. Hi, my name's Lee Arthur from Blenheim Chalcott. A question for you, Maggie. We think a bit about company market fit, not just product market fit, especially as we're building young SaaS companies. We feel that they can easily be roadkill for bigger tech, and I think that ChatGPT and others enables any tech company to move into any other space much easier. And I wondered if you had any advice in terms of competitive strategy for SaaS companies who are going to market to enterprise, but Microsoft, Amazon, everybody else, enterprise ChatGPT may come along and just add a feature and you're out of business.

Yes. So, uh, I think the biggest thing that a company can do right now is really differentiate themselves. And really think about how you build your moat. I do see, uh, in the side, I also, I, um, run a small fund with 7 other women, and so we are seeing every single day all these startups coming across our desk. And where these startups, where many of them are going wrong, is they are building a thin wrapper around an LLM. And you need to do something such as Sourcegraph, who is taking, uh, really rich data sets of code and best practices and fusing it with an LLM, and that's really what's going to set up that competitive moat. Whereas my advice to all startups is don't just build a thin wrapper on an LLM because there's a great chance that a large Microsoft, Google, whomever might build something competitive to it.

Yeah, just to chime in there, what we're starting to see are startups that are chaining together LLMs in order to like take much more complex workflows and simplify them, particularly across teams. And if you think about like the last 10 years of software, a lot of the software that's been built is let's take a feature of a bigger product and just optimize that. And now what we're starting to see is no broader suite strategy where it's much harder to replace, and it's not just a thin wrapper. Yeah, great answer. Uh, hi, thank you for doing this. The question is to you, Maggie. So I'm, I'm the founder of the startup called Talk to Data. So we are building Code Interpreter to work with not just a single CSV file, but across SQL databases and CSV files and stuff like that. So my question is, like, do you guys plan to release Code Interpreter API anytime soon? And the second one is, so what kind of integrations do you expect to see enterprises will be able to integrate into these complex data analysis tools going forward?

I unfortunately can't comment on OpenAI's roadmap whether we're doing it or not. I can't chat about that publicly. I'm sorry.

Okay.

Okay. Yeah.

Hi, Sharon. I got a question for you here. Here. Oh, I didn't realize there's a third line. Yeah. Thanks for speaking up. First of all, I mean, thank you very much for sharing. I actually just finished your fine-tuning lessons last week. Yeah. Awesome lesson.

Congrats.

I learned a lot. But at the same day, I actually read the news article from OpenAI that the Enterprise is released. So it got me a little bit confused. What, what is the difference between Lumni service and the Enterprise account for OpenAI? Supposedly it can address your privacy issues, enterprise privacy, data privacy issues for both of these solutions. So what's your ideal customer profile personas? Right. So, I mean, how— I mean, as a company like us, right, so how do we choose which platform that we would like to go?

Yeah, that's such a great question. So the way we view data privacy, and this is really coming from our customers, is that they, you know, they want to own their data. And as they have for the past several decades, they also view LLMs as an IP as well. They also want to own the LLM, which is really deeply intertwined with that data, and that includes the weights and all. So we don't own the weights of the LLM at all. All we do is help manage the infrastructure for it. And so when it comes to data privacy, this means that you can even— you can choose to do this on-prem, you can choose to do this on cloud, you can do this semi-cloud, you can be flexible around it, and you can build your models on top of that where we'd retain zero ownership over it, and you have complete control and personalizability on it. And that means when you fine-tune these models and build these models further on your customer's data, this means that that can always be owned by you. That can not be seen by us in any single way. Even if, you know, there's a leak or anything, nothing, nothing can make it so that we can touch that data or that LLM. So it's really about ownership and building out that moat. I know someone asked over there about, you know, thin wrappers. This is about building a much deeper moat where I think the future of those who will play in this space at all will be those who own their own LLMs. And that includes owning the weights of the model and everything so that you can actually manipulate them to do what you want the model to actually do.

One additional question about that is we do have developers working. I mean, there are Hugging Face communities, there are open sources. Uh, how do we decide to go to Lab9 or, or build something in-house?

Yeah, I think that's a really good question. A lot of our customers, um, are in this sweet spot where they have started to build out some of their own systems and using, you know, open, open source libraries just to understand, you know, how do I glue together all of this. Um, when you're ready for more of a production environment, uh, and a production build of this system, then folks come to Llama Nai because you can scrape together, you know, Hugging Face, DeepSpeed, all of these different great libraries out there, um, but it might just work for one run or a couple research runs, uh, but you can't actually integrate it in a production system and stay up to date. For example, um, we rewrote the PEFT library because all it was doing was, and this is the, sorry, the PEFT stands for Parameter Efficient Fine-Tuning, and the library out there is actually just, you know, the research group kinda just threw it up there. It's totally unoptimized for actual production systems. So we rewrote it and we just stay on the forefront of that and make it actually maintained for production use. And the idea is that this is our, like we're gonna optimize this, but we're not gonna optimize anything when it comes to specifically your data domain and your market. And the LLM is completely owned by you. But the infrastructure layer is agnostic across different markets. I hope that's helpful.

Thank you.

It's helpful. Thank you.

Hi, I have a slightly an extension to the last question, and this is a common question to both you, Maggie and Sharon. What kind of enterprises or verticals do you see building their own LLMs? Versus using ChatGPT Enterprise version? Yeah, I just want to be— because it's so confusing in the sense that last week, you know, we launched that we're building Enterprise. So ChatGPT Enterprise is really a tool for the internal employee. And yes, as I mentioned, we are absolutely building customization where you can bring in your company's corpus of data into that. But really, ChatGPT Enterprise is an internal employee productivity tool. And then you've got our API side of the house where, you know, I think everyone's probably pretty familiar with those. While we do have an enterprise program for the APIs, I just want to make sure it's really clear of like ChatGPT Enterprise versus APIs are actually two totally different products.

Um, also our customers, a lot of them actually buy both, um, just to be just very transparent about that. So they're often, they buy both. They want to be able to use, um, ChatGPT especially internally. A lot of folks Lam and I because they want to build these models into their products to have, to basically be a market leader in AI for, for their market. So they're building these as more externally facing, externally facing features. For example, one product, we work with a leading fitness company with about 6 million users, and they are building out, you know, this fitness app, and they want to be able to tailor this LLM to every single single one of their millions of users without spending, I think, over $1 billion in compute a month. So they're using us to be able to develop these models that are personalized to every single one of those users on a huge swath of data from every single one of those users. So that's one example, and that's an external-facing one, not in— I think ChatGPT Enterprise is internally facing specifically. And, and yeah, so that's— maybe that helps. I'm not, I'm not sure.

Thank you.

Hello, uh, I have a question for Sharon as well. So I'm the founder of Affai, where we are making this kind of what's called local-first software. They are web applications that completely run locally first. So as you mentioned, there will be elements for every industry as domain experts eventually. Uh, when will you think there will be even personal LLM that runs completely on your laptop? Do you think that's possible?

I think that's possible. I mean, we already see great work on Llama, uh, llama.cpp, right, um, out there that's running on these M1, M2 Macs. Um, I think 100% there will be personalized LLMs, uh, that'll be either running locally or, or on cloud, and that'll be hyper-personalized to every single person. And I think that'll come from any enterprise, because in any market you can imagine when you're shopping, let's say online shopping or something, there might be one personalized to not only your persona, but just you, just you directly. Or there might be one for fitness, like I said. There might be one, someone was talking about writing SQL, for example. So there might be one for each of those.

Yeah, yeah, yeah. That's what everyone would love to see. But to be honest, the open source model is by far not so mature as what we can get from ChatGPT. That's my personal feeling. So let alone that, we are talking about fine-tuning a personal one on the local devices. Do you think how long that will happen in practice?

Yeah. So I think fine-tuning, we've actually found to be able to outperform the top LLMs, open general-purpose LLMs on specific use cases, not general-purpose ones. So you can outperform GPT-4 or Claude 2 on specific cases that you want to actually do it in, as long as you have data. And of course, that is all— these are still hosted LLMs on GPUs. We also have seen this done on on-premise GPUs. I don't know. This, I think, gets a little bit closer to what you're saying. I think there is still work to be done between getting from those LLMs to the one that you want, which might be hosted on any kind of laptop. Is that correct? It depends on how much GPU.

Yeah, I mean, by GPUs, all of the startups are running for H100, right? While the majority of public are having it as good as, I mean, 2060, I guess.

Oh, yes. For fine-tuning, you actually don't need H100s to do a really good job. And they actually get peak performance on these models. I think that's a misnomer. If you want to build out the whole foundation model, that's— you need tens of thousands of H100s, but otherwise you don't need that at all.

Okay, thank you. Thank you. One more question. Uh, Thomas, uh, thanks everybody. I also have a question for you about the thin layer. I want to hear your, your take on how you do— how you pick early startups that they only have a thin layer about anything. What do you see? What do you look for in differentiations? Okay, so, so the question is, if you, you're going to build a startup that is just a thin layer, how do you build a venture-scale business from it? No, more, more thinking about it as most of these early-stage startups didn't have time to build something there. Oh, I think, yeah, everybody's thin anyway. Yeah, that makes sense. Okay, so one of the really interesting things about AI-enabled startups is their growth trajectories are meaningfully faster than a classic software company. Like their time to $300K or $500K in ARR is much, much faster. If you look at the open source traction, the total number of stars that they're able to achieve on GitHub, we've never seen a 0 to 1 year, 10,000-star open source project before. I don't know, there might be 10 this year. Just using LLMs. So assume you've got a company that's hot, that's a thin wrapper. The big question is like, okay, you're going to use that thin wrapper in order to do, in order to drive a ton of distribution, right? Like the goal is you're going to catalyze this audience, you're going to solve a very particular problem for that. And then at some point you have to be able to leverage that distribution to cross-sell other products or expand into workflows that are then more sustainable, like our audience talked about. It's really like, that's the goal, right? Anytime you've got a big technology discontinuity, all of a sudden distribution becomes much less expensive on a relative basis because people don't really know how to game the system yet. You can think about like the early days of Google or the early App Store days. It was the startups that figured out the distribution. These LLM-enabled applications, everybody's curious. And so as a result, people are really open to trying different things. And so if you can take that initial distribution from that wrapper and then use that to drive a bunch of distribution and create a moat, then, then I think you can build a sustainable business.

Thank you.

Thank you very much for the question. Awesome. Thank you, everybody. Big round of applause for our incredible guests. Thank you so much for joining me today. So grateful.

Thank you.

Thank you very much, everyone.