The Where, When, and How of AI with Theory Ventures, Open AI, MotherDuck and Lamini

Hi everyone, my name is Tomas Dunguz. I'm founder and general partner at Theory Ventures. Thrilled to be here with you all and our illustrious panel of guests. Today we're talking about how data and AI are changing the way that applications are built. We've got an incredible group of people here, starting with Jordan Tigani and Sharon Zhao and Maggie and would love to just kick things off by having them introduce themselves and we'll jump right into the agenda. Jordan, do you want to take it away?

Yeah, sure. I'm co founder and CEO at Motherduck. We're building an analytics engine based on open source DuckDB. I helped create Google BigQuery, led that for almost 10 or as part of it for almost 10 years before starting Mother Duck about a year ago.

Hi everyone, I'm Sharon. I'm the co founder and CEO of laminai. We're making it easy for enterprise customers to be able to fine tune their own LLMs. Most recently I was computer science faculty at Stanford teaching generative AI where I also got my PhD with Andrew Ng. I teach about a quarter million professionals online generative models, fine tuning LLMs. Most recently. I'm very excited to be here today.

Hi everyone, my name is Maggie Haught. I'm on the leadership team at OpenAI in particular overseeing the team that is bringing ChatGPT Enterprise to market. We launched about 10 days ago. Prior to this I spent two years at Webflow building out the sales team there and then prior to that six and a half years at Slack. From the very first founding sales hire out of headquarters all the way through acquisition to Salesforce.

Incredible backgrounds. Well, I'm really excited to talk to you all today about the way that data and AI are changing the way that applications are built. It's just been incredible last six months. Look at the rapid pace of innovation within LLMs, how they're really transforming a lot of the different ways that people are using software. And maybe Jordan, we can start with you at the data layer. You're commercializing DuckDB, one of the most popular databases now in the world. Super high performance in memory database. How are you seeing AI change the way that applications are built and the way that people are using data?

So to start out with I was a little bit of like a resistor thinking well you know, this isn't really going to change our world, maybe it's changing a lot of other things. But then I was talking to somebody who was giving a course on data thinking at Princeton and he was teaching people how to ingest data how to query their data and then how to visualize their data. But he wasn't teaching them any programming languages, he wasn't teaching them SQL and he wasn't using like higher level tools that can automatically kind of do this stuff for you. He was just using ChatGPT to generate Python code, to ingest the data, to generate SQL statements, to generate like the than the Python visualization stuff. And I'm like, wow. And this was a few months ago when ChatGPT had just been out and you realize that, yeah, this is going to change who can do this stuff and also change the skills that people are going to need and the tools that you're going to need to build. So I think that's one side of things and the other side of things is just the developers, people who are building things with that need access to data, are going to be changing how they access that data. I think the v1 is the vector databases, but I don't even think we know necessarily what V2 is going to look like because, okay, you get those vectors out, okay, what do you do with them? How do you actually access your data stores and relate that to what you're getting out of the models? So I think there's a whole big area of unknowns and exciting possibilities there.

Yeah, I mean, one of the things you touched on there that I think is really important is how much LLMs are democratizing technology. It's really much easier for someone who maybe not be as sophisticated with SQL to suddenly start asking and answering questions of their data in ways that would have taken them 3, 6, 12 months to learn beforehand. It's an incredible phenomenon and it's being adopted broadly within the enterprise. Right, Sharon? I mean you're seeing this, we're seeing this kind of across the board. You look at the total number of customers that are starting to use Microsoft's OpenAI technology has 3x, I think, in the last quarter in the reporting. How are you seeing enterprises start to adopt some of these technologies and how are they managing? There's a lot of technology here. It can be complex, there's tuning, there's lots of different components to it. What are you seeing in the market at Laminai?

Yeah, I think even just zooming back out at a technical level, I think actually public data is running out for all of what LLMs can take advantage of. So actually the next frontier for LLMs is in enterprises. And I believe the best LLMs for this next wave essentially will be enterprise LLMs and the enterprises that I'm Seeing that are at the forefront of this, they have started to adopt techniques around fine tuning their models to personalize models for every single user and every single customer of theirs where the data doesn't actually leak between those customers, but the model is highly personalized according to their access controls for data and just any kind of complexity around their data slices and use cases. So that's what I'm seeing and it's honestly really exciting to see this because this will further democratize all the possible use cases that can be out there far beyond what I can I know, even imagine.

Yeah, so, okay, so what we're starting to see, we're meeting lots, I'm a venture capitalist, we meet lots and lots of different businesses. They're starting to offer these software that's enabled with AI. And there's a question that we're sort of wrestling with in the ecosystem. Which is it the vendor who's responsible for actually optimizing and or is it, are there people inside of the business that are subject matter experts? Are you seeing any trends on which side is going to be responsible or mostly responsible for optimization?

Yeah, so there's definitely this tension between the two where AI expertise. It might take a couple hundred top AI researchers to be able to train some of these models such as OpenAI. However, I believe that, and this is based on my experience training these models, it's actually the domain experts will be driving the best models out there. It won't be people like me who can actually do all the model training, et cetera. All that infrastructure will be built out such that more people can actually define the trajectory of these models. And as we saw with ChatGPT, with more people being able to use these models, they thought of way more creative, different use cases than I in a silo with a few other AI researchers can think of. So I think the trend we're seeing is really the typical developer, the infrastructure developer, even the front end developer, all these developers who are able to access that data and be able to manipulate that data are the ones who can actually help develop the trajectory of these models and do the fine tuning and actually build out this next generation of LLMs for the enterprise.

Yeah, that's a great segue. Maggie. You're in the center of the world. OpenAI basically catalyzed big chunk of this movement. You recently launched OpenAI for Enterprise. Can you tell us what was that like? What are you seeing in the market? What sort of revelations can you share?

Yeah, absolutely. So I will actually break this down into three things. I'll share with you all a bit about our journey to building ChatGPT Enterprise over the last few months. I'll also share with you some of the learnings and kind of use cases and what we learned from working with our design partners. And then finally I'll share a bit about what is coming, which we're really excited about. So I was hired on to OpenAI in April, really with the mission of building out the go to market team for ChatGPT Enterprise. We knew it was something that we were going to build and that was because when ChatGPT launched early last December, all of a sudden we started seeing millions and millions of knowledge workers using this tool and every single day for their jobs. But then you have kind of the age old security, privacy IT side of the house that is saying, whoa, we don't want our corporate data being put into these models and trained on these models. So we put our heads together and said, let's build an enterprise version of this that has things like data training turned off, SSO, SCIM, I mean all the enterprise Y features, SOC2, CCPA, GDPR, all the acronyms. So when we set out to really launch this, what we decided to do was to select a pretty big variety of design partners. So for example Amgen, PwC, Bain, Samsara. We decided to go as wide as we could with different industries and use cases, quite frankly, because we wanted to see how healthcare uses this tool, how does banking use this tool, how does consulting use this tool? So we worked really closely with them behind the scenes of course, under some pretty mega NDA lockdowns for, for the last few months to understand what is it that the market wants to see for a V1 launch. Throughout this, there was a really big learning that came out to us and it was the importance of developing alongside your customers the idea behind really strong use cases enablement training. I think when we started off we just kind of assumed, oh, everybody knows how to use ChatGPT. But then we realized everybody uses it in a different way that they've kind of created themselves. So we've been thinking very deeply about how do we work with these design partners and now the world to develop very specific tailored use cases to their industries, to their Personas. ChatGPT can be used by every single vertical, every single company, every single department within a company. It's just mind blowing. So that was a really big learning for us, was really invest in customer success type work. Early on we set up Slack Connect channels, we asked these design partners to have weekly sync. We sent out surveys to all of their beta users. And so we feel really good about what we launched last week and we've also been quite public with our roadmap and what we're building here, which I think is also quite unique. Normally companies are a bit cagey and we've shared pretty broadly that the single biggest thing that we are working on right now is this idea of customization. So imagine being able to have your entire company's corpus of data or all of your own personal data infused in with ChatGPT Enterprise and basically being able to query all of my top sales deals, what were common trends across them and having that all come back for you. The other thing that we are also building right now, which is really exciting, is a business version, so more of a self serve version for SMBs. We realized we went out of the gates with enterprise but a lot of times SMBs just don't quite need all the enterprise Y features that we launched. So there's just been pretty huge demand from SMBs, from startups, from founders to be able to adopt this. The use cases are endless Code Interpreter, which is now renamed Advanced Data analytics is probably the most powerful tool I've ever seen in my life. You can essentially ingest in hundreds of thousands of rows of data or spreadsheets. I think about it as a super calculator. I use it personally for looking at leads, looking at sales data analysis opportunities, win loss rates, everything and it basically just synthesizes anything that would typically take someone a day to parse through in minutes and then down to the bare bones of like recapping customer calls, feeding in transcripts from Gong and having it analyze that and then write a next steps follow up email that is like beautifully perfect, personalized and tailored which would have taken me half hour per email before I can now do in 10 seconds.

That's amazing. I think the point that you hit there, that solution sale where you have to take something that's a general purpose, real innovation and really solve an end customer problem. That's what we're seeing in the ecosystem today. It's like people, we're really excited about the technologies and industry but we need to figure out okay, what is the problem that the end user really needs to solve. Sharon Jordan are you starting to see like, are you focused on individual Personas when you think about taking your products to market or seeing different use cases arise?

I think there's this sort of developer Persona and I think those, I think the way those, as you mentioned that everybody uses ChatGPT in a different way. So the developer is going to use it in a different way than an analyst is going to use it, which is different than a data scientist. And I think, yeah, there's going to have to be different tooling for each one of those.

We're hyper focused on the developer, so the software engineer, someone who can actually deploy a real app in production. And we're just really focused on getting these basically production use cases out. And we believe that the software engineer who sits most closely to their production apps will be the one to do that. And we've been finding that they're able to leverage data really effectively and efficiently. They're able to hook up their internal systems, whether that be recommendation systems or search systems, into these models as well. So they're, they're very capable at pulling together all of these different APIs and continually improving these models on a schedule. And so we're hyper focused on this Persona. But I'm also really excited to see just so many people be able to use this technology.

Yeah, that's been a big change. I mean, 10 years ago, the way that I thought about the data stack was you had systems that produce data and then they would puts it, put it into like cloud data warehouses or data lakes. And then there were three consumers, there was the BI use case, exploratory data use case, and then there was machine learning. You're trying to build data products. And now it really seems like those machine learning engineers or that functionality is becoming part of the core products, becoming part of the core workflow. Right. Like Maggie talked about gong analyzing those sales calls, that's part of the product. And so the machine learning engineers are now starting to be put into the path of production. And there's this blend, there's this mix between, okay, you've got a classic software engineer who's used to shipping code with like CICD in the classic workflow. And then you have machine learning experts and bringing in different technologies. Right. Whether it's like data caching or customized models or off the shelf APIs and you're putting it into production. It seems like there's a merger there of two Personas that historically have been separate. And the landscape is sort of rising up to how do we enable those people to be as effective as they can be using things that maybe their expertise is sort of out of their domain. So you need a higher level of abstraction. Is that what you're seeing in market, Maggie?

Yeah, absolutely.

Yeah.

And I think that's incredibly important actually merging those two together because historically maybe to give a fun anecdote, one of my friends used to lead engineering at Uber. And so a thousand person engineering team, and what he found was the machine learning team was building models in Python notebooks that were obviously then had to be sent to the production team to then ship it into all of our apps. And one time, because Uber was moving very fast, let's put it that way, they had just fired the person who ran this other model, so they had to literally go find the person's personal laptop, open up the Python notebook, rerun the data on these few states where they had to have an updated model, and then somehow rip that model out and thank God they found that laptop and put it into production and now it's shipped on our phones. And that is an insane process that's super brittle. And I guess another anecdote is just it's really hard to get some of these systems up. And in machine learning, a lot of AI people are like, yeah, we push the performance of this model and we define performance as accuracy or accuracy along these general benchmarks. Maybe it's sixth grade science questions or something, but a production software team will look at that and say, that's not performance to me. I also care about the speed, the latency of this API call. You can't give me a better model and give me 100x slowdown of this model because I'm putting it in an autocomplete type of interface. That's not going to work. I think merging these teams together, bringing them together is really important to actually shipping these models in production. I think historically it's largely been this R and D team with a really thin line to production and oftentimes failing that. Only very few teams could succeed at that before, but I think now many more teams will be able to succeed at that. And it's just really exciting for AI more broadly.

And I think as those teams are merging, there's this sort of interesting debate, which is what is the lingua franca of what language are people going to be writing in? Is it Python or is it English? And so as people start to be able to use just English natural language to do some of the tasks that were in the other camp. I think that also can help accelerate that merger, even if sort of maybe fully getting towards, you know, you describe your program as sort of like an English language paragraph and have that turn into a production system, maybe that's a little bit further, further off. But I think as you start to kind of peel off workloads, it can just Help those teams merge.

That's the thing that's so exciting when you think about Microsoft Office. 330 million paying subs, whatever. Let's say a third of them use Excel. All those people want to be able to access some more sophisticated data within the organization. They want to analyze it. Like you said, like historically they've really needed to learn either like VBA or SQL. It's a language not everybody really knows. But now if they can start to speak English, you really have sort of the power of data now being exposed to, I don't know, a 10x100x greater the population. It's pretty incredible. So just as we've seen, I mean, what we're talking about on the technical side of an organization, you really see a big transformation in what's happening. Maggie, you're a sales leader. How are you seeing LLMs and these applications change the way that companies are starting to go to market? What's going on in the front of the house?

Yeah, I think it's best to start with a bit about how sales has changed over the last few years, because every year it seems to be there's a new step. And I think the short of it is that AI is going to fundamentally change how the sales role operates and looks. And it already has. We're already well into that. 2017, 2018, it was pretty easy to prospect into companies. You know, prospecting is kind of the bread and butter of any sales organization to generate opportunities and customers. Then all of a sudden you have these tools of sales loft and outreach which come out and just take prospecting mainstream. So then all of a sudden, sales reps are sitting there saying, what do I need to do next to get in front of buyers? The next step was personalization. So years ago, personalization wasn't even a thing. Last few years, it's been all about, how do I research this company, this customer, this person. Did we go to the same school? What can we kind of connect on together? Well now LLMs have made it so research and personalization can be done in less than a minute. In less than a minute, you can have ChatGPT or Jasper or any of these other tools that are out there right now. Go write an email that sounds very personalized and tailored, but was actually written by an LLM. So now this kind of next era of sales and sales orgs has to really be thinking deeply about how to interact with the customer. No longer can you get on a call and ask somebody a million discovery questions. You have to show up to that Call with a strong point of view with how your product is going to save this company money or solve their pains or whatever their mission is. And the beauty about LLMs is now they can help you do that research in advance. So back in the day when I was sales rep, I used to spend two to three hours before a big meeting, even more researching and prepping and practicing my discovery questions and coming up with a value hypothesis. Now that time can be significantly reduced to maybe 20 minutes max and you can literally have the LLM be role playing with you of the questions that you are going to ask and the value statements that you're going to bring. And Now I have LLMs also on the other side of it are going to save so much time for the sales professional in terms of administrative tasks. No longer the days of writing, as I mentioned, long follow up emails or logging your notes into Salesforce. You can literally feed, as we talked about, transcriptions from gong directly into there and use prompt engineering to write these beautiful summaries. There's all the new tools that are building slide decks for you, which is wild. So you're able to save on the administrative side of things. But the sales rep in this day and age and sales teams have to really step up with their value selling.

Do you think that changes the structure of a sales org at all or do you think we stay with like the SDRA model?

I think we stay with it for now and then I think AI, I think sales teams and companies that adopt AI are going to significantly outpace those ones that don't. So I actually think now like the common sales rep should be doing everything they can to actually learn prompt engineering, to learn how to leverage AI in their day to day to make them so much more efficient. I think maybe someday the model changes, but I don't think we're at that day just yet.

So we're all salespeople, right? If you're head of sales or you're a salesperson or you're a venture capitalist, we're all selling something. These tools, like you said, they open up like enrichment discovery. How are you using LLMs in your day to day in order to improve your life? Are there tools that sort of stand out for you where you've seen a complete game changer? Anything kind of come to mind?

Are you asking me?

Yeah.

I think certainly people that are interacting with data, it's changing their lives. One of the things I do a lot of is I do a lot of slide decks and things like midjourney and generative AI make it so much easier to have the right image. And yeah, maybe sometimes people have too many fingers or there's ducks without wings or kind of wonky eyes. But I think as the technology gets a little better and perhaps the operator gets a little better at prompt engineering, those kinds of quirks will go away.

Well, with mother duck, I mean, you've done an incredible job making that such an iconic part of your brand. I can imagine getting the duck anatomically correct. Really, really important.

It is important.

I think one thing that we were analyzing our data when it came to all our marketing blog posts or tweets, everything going out and there was a 10x increase in anything with an image. It was just much more engaging for people to actually have impressions on. We found that just spending some time on midjourney was worth more than spending time on the blog posts. And in any single way, like hands down, just like spend a few, a few extra minutes here. Prompt engineering or really just like generating an image of your choice and clicking through them versus actually writing out the blog posts that no one's going to read. So I think that was a really big lift for us when it came to marketing specifically.

Yeah, just piling on those images are super helpful. I was unfortunately sued for using an image where I didn't have the rights. And so I learned a very important lesson. And so now most of the images on the blog post that I write are midjourney generated because now I can be confident that I'm not going to get myself into legal hot water again with rights issues.

Yeah, go more into that story.

I'm kidding.

Well, so I mean, you know, I was like, I was, you want a hero image, right? Like you look through like the click through rates and the dwell times. And so I was using Google and there's a filter on Google where you can search for things that are Creative Commons license. And so I found an image on the Google search results that ostensibly was licensed with Creative Commons. Turns out actually somebody owns the rights to the image, which I was notified later. And so now at least with like a synthetic image you can be totally confident that effectively I don't know what the latest legislation is, but nobody owns the image or I at least on the the image. So that's been a huge help. Do you use anything daily, Maggie? That really sort of where you see like a pretty meaningful improvement in your day to day?

I would say ChatGPT Enterprise, but that's not a fair one. I have been the last few years. There's just no tool that's better than Gong. Gong is a sales leader. Gong is getting ramped up as a sales rep or a new hire. Gong for product. I mean, as we've been building ChatGPT Enterprise, we would love to have our product leaders on every single call. It's just not realistic. So we can literally take snippets out of Gong and share all the common feature requests. We can drop in hashtags. I mean, I'm on Gong's website on like from my previous company of speaking about them, but there is just no better tool out there than Gong. So I'd say ChatGPT Enterprise, Slack and Gong are the three tools that I use for many hours every day.

Yeah, that's awesome. We do the same thing when we meet founders. We record the conversation so we don't have to ask them to repeat. And we're starting to save all that information, produce LLMs on top of that summarized diligence. And we produced our first investment memo summary from an LLM. And so we can just start to see like, as we gather more and more of this information, we can accelerate some of the things that are wrote within the org. The other thing that's been really useful for us is blog titles. So, you know, I think about like, LLMs, I mean, there's the incredible things where they're taking the sum total of like human knowledge that we've put on the Internet or the data sets that we've trained, and then they can kind of replicate them. And you think about all of the amount of human effort that's gone into optimizing blog titles. That's all on the Internet, right. And it's all encoded within these LLMs. And so if I take like a blog post that I've written and then I ask one of these models to produce like three or four suggested blog post titles, I actually get three to four times better click through rates on the titles because I'm aggregating all that knowledge. So that's been a big, big help for me.

Awesome.

Yeah. Okay, so to bring us home, like, quick take, where do you see for your particular company, like, where will we be in three years? How will LLMs have actually changed your business?

So I'm going to a little bit of a cop out on this one. And it was an answer that founder colleague of mine gave, which is before ChatGPT, he could sort of look at, look five years out into his business and be like, okay, I can see the road and I can see like, you know, where we're going and I'm pretty sure, like we're going to be successful and then, and then like ChatGPT comes along and like this cloud comes over the road and you just, you can't really see more than five feet out in front of you because, like, you know, as, as, you know, OpenAI may release something that, you know, makes something totally irrelevant or make, make, make it so that developers or analysts start to use some, some sort of other, other technology. I think the only thing that I think is clear is that people will be doing things differently, their jobs will be different. I think, for example, the data space has sort of started to aggregate into these different job titles. Analytics engineer, like somebody who uses dbt, data engineer, somebody who munges data analysts, et cetera. But kind of as all of that stuff can be done with prompts, I think that there's going to be another. I think there's going to be a reshuffling of those roles.

I think every enterprise will have their own LLMs. And I think what that'll enable is in every single market there are domain experts. For example, like sales, they're domain experts. And what an LLM does is it becomes a scalable domain expert. And so for that market, there will be scalable domain experts for every single type of expertise, tailored to every single type of customer and user. And so I find that really, really exciting. So I think that'll dramatically transform job roles moving forward and that'll dramatically change basically how businesses need to operate and who they need to even bring on and hire.

Maggie, in 60 seconds or less, predict the future for us.

I can't comment on OpenAI's roadmap, but what I can say and what something that I'm the most excited about in terms of an industry that is going to get completely upended is education. Just last week I was with the CIO of ASU and he has these huge plans to bring AI tutor bots and AI coursework and really democratize the future of education so that anywhere in the world has equal and equitable access to education. And I think that's going to be one of the biggest impacts this world has ever seen. Is everything that's going to happen with AI and education, it's going to be just absolutely bonkers.

I could not agree with you more. I mean, I think the, the thing that we've noticed across a bunch of different companies is if you have somebody who's new to a space, new to domain, and that could be math or calculus or biology, it could also be SQL or data engineering if you can give them access to an LLM, the quick feedback loops and a super patient instructor where you can say like, I don't understand it or show me this or. And the other day I had written a line of Bash code and I had forgotten what it was. Like ChatGPT, like what? What was it that actually wrote in this piece of code and explains it to you? It's an incredible, incredible tool. Awesome. Thank you everybody. Big round of applause for our incredible guests. Thank you so much for joining me today. So grateful.

Thank you, thank you very much.