513. Harnessing AI and Experimentation in Startups feat. Jeffrey J. Bussgang

Unsiloed Podcast is produced by University fm, elevating the stories of your institution.

Welcome to Unsiloed. This is Greg LeBlanc, and I'm here today with Jeff Buskang, who is a senior lecturer at Harvard Business School. And also, I don't. I don't know which one's your main job. Right. Are you a lecturer and a venture capitalist on the side? Are you a venture capitalist and a lecturer on the side? Because you're also the GP and founder of flybridge Capital.

Yeah, I'm not sure which either. Mostly I would say I'm a venture capitalist and a lecturer on the side.

Well, that might be true if all you were was a lecturer, but, I mean, you've written over, I think, what, 80 cases? I mean, something like that. Some crazy number of cases for Harvard, some of which I've read and used. And you've also written a couple of books, right? You got this one called Mastering the VC Game, got one called Entering Startup Land, which is kind of like a career guide. And then this most recent one is called the Experimentation Machine. So welcome, Jeff.

Yeah, thank you. Great to be here.

Well, I really liked how you kicked off the book by saying this recent book, the Experimentation Machine, by saying it's about timeless methods and timely tools. And so it seems like being a founder and being an investor, there are certain things that don't change. There are certain things that they were true back in the early days of EC. They're going to be true probably in 30 years. But there have been these new technologies and tools, and in fact, every couple years, there's new technologies and tools, and you have to be on top of them without forgetting, of course, these tools, timeless rules. And I think the other thing I really liked about the book at the very beginning, and hope we can dig into this, is that you said that a startup and this building, I guess, on Steve Blank's work is effectively a laboratory, and the CEO of a startup is the chief Experiment officer. And the reason why I like that is because I've been using that term and I thought I came up with it, but I guess it's just one of those things that's common knowledge in the world of venture capital. So I guess the question I would have for you is you say that this is something that can be done using a scientific approach, starting a company, launching a tech venture, and yet there are a lot of people that seem to think that the founder, the entrepreneur has this special, you know, charisma, this special power, this special vision, and it's something which is radically different from what we might think of as the domain of science, something that can be taught, something that can be learned in a Harvard Business School classroom. So to what extent is the starting of a new venture a scientific enterprise, and to what extent is it some kind of artistic and creative enterprise?

It's definitely Both startups are highly non deterministic and so there's really no single playbook or single formula. And yet there are, as you noted and as I cite in the book, timeless methods that you can apply to improve your odds of success. Now, you're never going to guarantee success. It's not an engineering formula where certain inputs result in certain outputs. There's just too much randomness and as I said, non determinism, non determinism out there. And every age and era and context and startup and individual are so radically different. So it's highly unpredictable. Yet as I said, timeless methods. And then as you noted and I write in the book, timeless, timely tools. I mean, the tools, they're not just getting better every year, every month, every week, every day. Just as we stand here today, the release in the last few days that OpenAI has made around deep research and agentic capabilities and chain of thought advancements are just mind blowing. One of my colleagues claims it's the most significant launch since ChatGPT two plus years ago. And that's just one company, one product. And we've seen company after company launch breakthrough product after breakthrough product at a dizzying pace. And so the challenge for founders is to adopt these incredibly powerful tools to be more effective, more efficient. What I call 10x founders. Drawing from the mythical 10x developer notion that we have in startup land and yet still adhere to the timeless technique, timeless methods of value creation.

Well, I think you, you referred to a blog post that you published back in 2012. I think it was where you said, I don't know, was it tongue in cheek or was it sort of, you know, heartfelt belief that every startup needed to have a machine learning PhD on the founding team? And I remember this was McKinsey was also talking about how there was this huge bottleneck at the time, right? And there was going to be a huge shortage of data scientists and machine learning experts. And now you pulled back from that. Now is that because the technology has become sort of more accessible or is it just simply that the technology is changing so quickly that you don't even have time to become an expert? You need to general.

Yeah, it's more about accessibility and the quality of the platforms. When I was a computer science major in the late 1980s at Harvard. I had to learn assembly language as the language of choice to help program computers. And then of course there was an abstraction layer and an abstraction layer and an abstraction layer. And now we have all these no code, low code tools. And then a further abstraction layer comes with the generative AI capabilities where the new hot programming language is English. So it's extraordinary to me how incredible the tools are that we can use to build prototypes and real full applications just through conversant natural language. And so my point about in 2012 I was saying machine learning was the wave of the future and everybody needed a machine learning PhD or engineer in their startup. Now we just need really great applied logical strategic thinkers who are 10x founders and know how to use the tools and build that experimentation machine within the company to test the tools and run the experiments to try to find product market fit as fast as possible.

Now I teach mostly courses on strategy, right. So for you know, mature, semi mature companies. And then every now and then I'll switch over and start teaching in areas related to startups and venture. And I mean I think that obviously there's commonalities and you highlight how every business, whether small, large, early or late, has to have these four things. Customer value proposition, go to market, infrastructure, monetization strategy and so forth. But for the startups, they don't know what they are. It's a process of discovery and exploration. And so I mean that process is, you have to be good at that process. I mean that's the job of the founder. To, to what extent can founders that have never done this before actually do it and figure it out? And what, what is the role of the venture capitalist? I mean, is the venture capitalist kind of the, the, the coach, the one that helps them to figure out what the priorities and the sequencing of the experimentation ought to be?

Well, first I'll go back to something you said about can you be taught to be an entrepreneur? I've been teaching at Harvard for 15 years in the MBA program. The second year MBA is the selective called Launching Tech Ventures. And I've had over 2,500 students take the course. And I will tell you, entrepreneurship can be learned. We have students coming from all over the world and all types of industries with very little startup experience. Just today I was talking to a student earlier who came from GM and she's had a fantastic five year journey in the AV unit of GM as an engineer and product manager and she's in the middle of launching a startup. And she's using the tools and going through the process to develop her value proposition and do her ideation and test her hypotheses and prototype and bounce ideas off of her target customer and discover who that target customer might be. Just what you were saying, that search and discovery process. Chris Dixon has this great phrase, the idea maze. Going your way through the idea maze to find the big idea, that is absolutely a scientific process. And as I said earlier, you can't say for sure predictable outcomes, but you can say that you can improve the odds if you adhere to a rigorous process. Now, all that said, there's also a class of founders who don't want to go through a process who just have this stroke of brilliance, this idea. Maybe it's from their domain knowledge, maybe it's themselves as a user. As we've seen time and time again, amazing companies can be built that way as well. And I think one of the lessons from my teaching and from my 20 years in venture capital is that there's all different paths to success. All, all different types of founders. Founder market fit comes in many flavors. Ideation has many paths. And yes to your last point of your question, the venture capitalist who decides to work with founders when, when we make those investments, we try to be that coach and advisor and consiliary to help that founder go through that idea maze and walk that path.

Yeah, but I think you said something that I found really interesting, which is that for a startup, the strategy is really all about test selection, Right? So it's about trying to figure out which of the almost infinite number of experiments that you could run is going to yield the largest amount of uncertainty reduction or information gain. Right. So how does one know which experiments one needs to run? Like, how does one know which ones are going to actually know, yield the biggest benefit, and answer the most important questions.

It's the, that's the billion dollar question that the founder has to figure out. And with the tools now, we can run so many more tests. And so this question of test selection being strategy is really, I think, the essence of what founders need to think through. Because anytime you have an organization with limited bandwidth and a limited envelope of resources and capital, you need to make prioritization decisions. You need to focus. And so what I advise founders, both that I teach and also through my flybridge investment activities, is that they should select the tests that are going to uncover the most controversial part of their business model and have the highest likelihood of leading to a valuation inflection point if the test is successful. So for example, you may be in a market where you don't need to validate demand. Demand is obvious. Maybe. For example, it's a company that I wrote a case on called C16Bio, which is a synthetic biology company focused on synthetic lab grown palm oil. If you can grow palm oil for the same price at the same cost, quality performance food companies are going to take organic lab grown palm oil over palm oil that comes from fields because of the environmental impact, it's a commodity. So you don't have to validate demand. You have to validate that you can manufacture the stuff and get down the techno economic curve and hit the price point. So don't waste time doing a lot of customer discovery around demand validation. Spend your effort on getting getting into the lab and seeing if you can actually map out a pathway to getting down the techno economic curve. So it's whatever the key issue is that will unlock that valuation inflection point is the test that you need to focus on first.

There's no single template which dictates the sequence, right? The who.

No, I do like focusing on the customer first and the whole. And that is a formula that I have found most effective. Because why spend a lot of time building until you really understand the problem. Do your deep customer discovery, figure out who is your ideal customer profile, figure out what's that sub segment of the subsegment that really care about what you're building. And so really lock in on the who and then really lock in on what. We refer to in our industry as a hair on fire value proposition. There's nice to have, there's must have and then there's hair on fire. Like it's a top two or three priority. I'll drop everything. If you can solve this problem for me, if you can get that nailed, then you can build the what and you can build the company from there. But there are also tech driven companies that start with the what, like my example of the synthetic bio and synthetic palm oil company. And then they have to go figure out the who and the how. So there is no exact formula. But I just generally believe if you're not technology driven or science driven, tough tech ventures oriented. Focus on the who, figure out the really the pressing burning heron fire value prop and problem and then try to see if you can solve that problem.

Yeah, it seems like engineers oftentimes start with the what. Right. So I remember when blockchain was a new thing and I had all these engineering students that I was teaching and they were so excited about it and I was like, yeah, but, you know, what are you going to do with it? They had to try to figure that part out, but that seemed to be less interesting to them than the coolness of the tech. Yeah.

One thing I talk about at HBS and your colleague Scott Brady at Stanford Business School also talks about, and we collaborate and he comes into my class, as he did just last week, is that really smart, capable entrepreneurs jump to the solution and they cut off the problem exploration process too soon and they don't have the patience to really dig into. Did I find the biggest, hardest, most gnarly problem? That really is the stay awake must have hair on fire value proposition problem. And then let's narrow that down and then let's work on the solution. But a lot of really bright people, they just jump right from A to B to Z and they don't want to take the time to explore the other paths in the step process. And sometimes as a result, they spend all this time building a company that has a flawed fundamental problem, which is nobody really cares about your crappy startup. Nobody cares about your value proposition. It's a nice to have. If you put it in front of me, sure, I'd give you positive feedback, but if I'm a busy person and there's inertia and there's clutter and there's a ton of things coming at me, I'm just not going to pay attention. Yeah.

It seems like the choice variables that are available to the entrepreneur include the sequencing of the experiments, but also speed. Right. So you talk later in the book about scaling and how some companies scale too quickly, some scale too slowly. And so, I mean, how does one know the right amount of speed? Obviously things are becoming much more urgent. You have to move, you know, more quickly because everyone's moving more quickly but you, as you say, you have to be, be patient in some ways. And I really like the part on technical debt.

Right.

Because it seems like that the optimal amount of technical debt is not zero.

Right.

Because if you, if it's zero, it means you're moving too slowly. So, so how does. Should one be thinking about, you know, as one's building the company, you know that you're going to be creating future problems right along the way. How do you know which of those problems are worth taking on and which ones you need to invest early to make sure that you don't have to deal with them later?

Yeah. As you may recall, in the book, I articulate four or five criteria that help you decide whether you're well suited for blitz scaling, as Reid Hoffman describes in his book, or more of the test and learn cautious lean startup approach, as Eric Ries describes in his book. And there's probably more examples of premature scaling failure than there are of companies that failed to scale fast enough and seized the moment and lost the window. So on balance, I tend to encourage founders to just be a little more cautious and deliberate and thoughtful in their journey. And yet there are also certainly plenty of examples. And in the AI category, we're living them where it is. It does make sense to scale aggressively and blitzscale. And so the four or five criteria which you'll see in the book are, you know, is do you have strong network effects? Do you have strong product market fit? Do you have strong availability of capital at an inexpensive price? That's something that happened in 2020 and 2021. That window closed in 22 and 23. And and further, do you have a competitive dynamic in the market where you can't afford to take your time to work your way through? In an enterprise software market, no one's going to capture the entire market in a year or two. You have a little bit more time to be deliberate and be thoughtful about your maturation through the technology lifecycle adoption curve and getting the various segments between the innovators and the early adopters and the early majority on side. In some business to consumer market, somebody might capture the market. And if there's a strong network effect and if there's strong product market fit from an alternative and there's a strong capacity to raise capital, somebody may leap ahead of you and seize the moment before you can capture it. We saw this a little bit in the sports betting space with these two titans, FanDuel and DraftKings, just blitz scaling their way through super bowl ads. Since we have the super bowl coming up in a couple days, or at least at the time of this recording, when people listen to this, they will have already known the result of the Super Bowl. But these ads and this blitzing and, you know, we got to capture this market because there's only going to be one or two winners in this market. That's a rare situation. More likely than not, you have the ability as a founder to take your time, pace your way through that process, and really make sure you're iterating your way to success.

Now is, is that part of the job of the venture capitalist to kind of, you know, whip the, you know, whip the buggy and then kind of apply the brakes? And is the tool, I mean, is it really funding that's the tool, right? You inject a whole bunch of money and that speeds things up. You maybe inject a little less money and that slows things down. How do the venture capitalists influence the speed of scaling?

I don't love the with the buggy metaphor. I think it's derogatory, maybe to founders, but I do think that venture capitalists as coaches and support networks can advise very effectively on pattern recognition and say, look, do we really have product market fit if we make this investment? Will it be a high ROI investment if you add five more salespeople, do we have high confidence they're going to be productive or should we add two or three salespeople and make sure we get our sales training muscle built up and make sure that the marketing air cover is there and the customer case studies and references are there to then pace our way to adding more and more salespeople? Is the market adoption curve at the right moment in time? What are the signals that we're seeing in terms of tailwinds? We're a little company, we can't do it all on our own balance sheet. We need certain tailwinds and secular trends to support us. So yeah, I think we as venture capitalists, as board members can be very helpful and valuable to founders to be sounding boards to help gauge these pacing decisions. I just had a board meeting last week and this is, you know, January is board meeting month and Everybody's setting their 2025 plans. And we asked the question of the founder, you know, what's the thesis of the investment pace? What's your theory of what, you know, out of our balance sheet, how much should we be investing this year in growth versus saving it for next year? And just in terms of our readiness and the market readiness in this moment. So those are the types of questions we ask all the time right now.

It seems that the gen AI, I mean the biggest impact is that it just dramatically increases the ROI on your experimental dollar. Right. And you can scale without growing. You said. And a lot of my VC friends talk about the one person billion dollar company. It's like, you know, you just hire a bunch of agents and put them to work and kick run your experiments and bang, you got a one person unicorn. I mean, is this just talk or is this actually where we're headed?

So I'd say in the book, and it's Sam Altman who had that original quote about the billion dollar unicorn. I say in the book, I don't know if that's going to happen anytime soon, but the 100 person company that reaches a billion dollars, the 50 person company, the 25 person company. I think that's happening in the next couple years and I certainly think it's happening that we will see companies that hire and recruit and manage more AI agents than humans. And I think it's certainly going to be the case that we may be hitting peak employment at some of our largest tech companies as they continue to do more with less and get more and more efficient as users of their own tools, eating their own dog food. So we, we have companies in the flybridge portfolio that are achieving extraordinary results with fewer people than we ever would have imagined. And it's not that they have AI replacing sales, it's that the sales leader and two or three sales team members have dozens of AI agents who are helping with lead qualification, lead nurturing, customer success, renewal, usage, behavior, et cetera. It's not that they have only one engineer, they have a dozen engineers. But the productivity of those dozen engineers when they're using code development tools like replit or Codeium or Cursor are extraordinarily more effective and efficient. So yeah, I think it's really an exciting moment for productivity in these companies that are really adopting these modern AI tools.

Well, I mean, does that mean that the hit rate is going to go down? Right. So you know, we know it's a hit driven business. We know that VCs, you know, lose money on the majority of, of their investments and they're counting on a small percentage to, you know, make up for that. I mean, are we going to then see a much smaller percentage that, that actually make it? I mean someone would say that if these companies are using these tools, these tools are commodities. I mean, everyone has access to the same agents, everyone has access to the same LLMs. And so if somebody's trying to do some idea generation, you've got a hundred people or a thousand people or 10,000 people, 10,000 funded startups that are, you know, running the same experiments, doing the same ideation, doing the same, you know, brainstorming with their LLMs. Is this just going to make it that much more competitive?

It's super competitive. But I'll also point out that the outcomes are way larger than they ever used to be. When I was an entrepreneur, my company went public in 1996. We had a billion dollar market cap on our first trading day. That's a financing this week for OpenAI. At the time we were one of the largest companies Alien Lee famously described the Unicorn club. And it was a very Elite club. Today there are hundreds and hundreds of unicorns created all the time. 10 billion is yesterday's 1 billion. Today's 100 billion was yesterday's 10 billion. The bar just keeps going higher and higher. And now think about our comment and dialogue about agents. We have a legal AI company called Noetica and they're not going after the IT budget of law firms. They're going after the staffing budget, the hourly work of the lawyers. And there are plenty and plenty of companies in our portfolio. Melody Arc is another one in the customer service space and in other people's portfolios that are not just going after these tech budgets, but are going after the entire budget. And so the outcomes, I believe are going to be even larger still. So yes, more competitive inputs, but I think even more extraordinary outputs are coming in the next few years. And you're already seeing the early signs of that with the first trillion dollar companies, the first multi trillion dollar companies, the first startups that reach $100 billion in a few years. It's really extraordinary how much value is being created.

Yeah, so this new universe of agents, I mean, you know, they're, these are not the open AIs, these are not the big foundation models. These are specific applications in specific industries. I mean, how do they create moats? Because it seems like, you know, if somebody says, hey, I want to be the, I want to automate discovery. Well, I mean you might have a hundred different companies offering automated discovery. So where's the differentiation? I mean, if they, if they do they need to get some kind of proprietary data, do they have to develop some interface sticky? I mean, how do you, how do you differentiate in this world?

Yeah. At flybridge we talk about three sources of differentiation for AI application companies. One, as you hinted at, is a proprietary data set. So for example, one of our imagine.

Like Lexus, you know, Lexus decided to get into this business, they could kind of choke off access to certain information and then they'd sure have an interest.

I'll give you the example of Noetica, the legal AI company. They're ingesting the multi hundred page corporate finance legal agreements that law firms have in their archives. OpenAI doesn't have any access to those legal agreements that specify the terms of private credit financings and they match that with public private credit terms to generate an extraordinarily rich terms database and help lawyers and companies negotiate and complete contracts for private and credit financing. So there's sort of a set of proprietary data that aren't available to the public that these AI the best AI companies will have access to. That's one, that's one moat. The second mode is to become the system of record with the last mile workflow. And that's the ability to be an environment in which the user is in that environment all day, every day. So AllSpice is one of our portfolio companies that is the GitHub for hardware engineers. So it has both the collaboration cloud platform and the AI Copilot for hardware design. And their users are in AllSpice all day, every day building design. Just like software engineers are in GitHub all day, every day building software. So second, building that system of record and those workflows. And then third is that interface. The human AI interface I think is really, really important. The quality and speed with which we can solve problems as humans is. The capacity is getting greater and greater. But we need great interfaces. Voice is an exciting new interface that we're all experimenting with. I'm sure many of your listeners have been playing around with the voice capabilities of the gen AI platforms like ChatGPT. I created a clone of myself using Delphi AI. So you go to Delphi AI Busgang and you can chat with an AI clone. Not only.

I should have just interviewed that, you.

Could have interviewed my clone and I could have been on the ski slopes today. So that's I think a really powerful new interface methodology. So I think there's really some exciting new modalities. But that's another important vector of, of competitive mode is the, the, the interface with the human, so proprietary data system of record workflows for that last mile and that human AI interface.

Yeah. I mean it would seem though that the vertical SaaS companies, the incumbents would have an advantage. Right. So if you're salesforce and your, you know, workday and servicenow, like you already have the customer and you already have a bunch of data, right. Or if you're you know, contract management software platform or something like that. So how are these new entrants getting into the workflow?

So every technology shift, the incumbents say to us, I've got this covered. I'm the gorilla in the market. Don't try to compete with me. Twenty years ago, Steve Ballmer, when he was running Microsoft, used to invite me and some other VCs to the valley to get a briefing on the Microsoft roadmap. And he would point out all the areas that Microsoft was going to be the dominant provider and try to scare us away from investing in any of those areas. And he would point to little corners on the market map and say this is a little white space you could build on our platforms and maybe build a nice little niche company here. Look, when we invested in MongoDB 15 years ago and led their Series A, we had a thesis that the shift to Web apps and NoSQL would create space in the database market, despite how strong Oracle was and IBM and Microsoft were in that market. And lo and behold, MongoDB is a 20 plus billion dollar market cap company with a massive foothold in the database software market. So we are very convinced that there's plenty of white space for AI native companies to enter into these vertical SaaS markets at the same time, just as was the case if you look at the Oracle stock price over the 15 years that MongoDB has created $20 billion of value, Oracle stock has gone up tremendously, as has SAP, as has all the legacy software companies. So we do have a situation where we do have multiple winners in the market. The legacy players are going to do really well as they add bolt on AI capabilities into their suites and their entrenched customers are going to enjoy those capabilities. And new entrants are going to do really well also because they're going to come in with AI native applications and they're going to find breakthrough, rather large niches in the market, just like the MongoDB did.

Yeah, so it's more like the startups are complementing rather than completely substituting for these.

Yeah, or I would say the market is growing so rapidly that you're going to see multiple winners. There's going to be some complementarity and some substitution.

Well, you did talk about the use of channel partners, partners in general, and you said this can be a fantastic opportunity or it can be sort of a handcuff for a startup. I mean, how should they be thinking about partners?

I'm a big believer in the early days of a startup being all about maximizing learning, not maximizing metrics. And anytime you work with a channel partner, you're allowing that partner to get in between you and your end customer. And you're therefore stilting some of the learning and nuance of, you know, what the needs are of the customer, how they like to buy and what problems they're trying to solve. So my general advice is don't allow any gatekeepers to come between you and your customer in general, have direct contact and over time, as you learn about the customer needs and learn about the sales playbook, you can move your way through the sales learning curve and then eventually develop channel partnerships. All that said, there are exceptions to every rule. There are some markets where you need a channel partner to get into that market. But generally speaking, my rule of thumb is no gatekeepers. Stay direct, maximize learning.

How do you measure learning? I mean, they say you can't change what you can't measure. And in the hunch mnemonic that you provide in the book, I really like this. It brings together in one spot all the key metrics of success of these companies. But those metrics don't track learning. Right. They track kind of where you are. Right. So what's your nps? What's your churn? What's your LTV to Keck? But how do you measure how much more you know about these things? Right. So you could have some loosey goosey number and then you could have a really precise number. And if you go from loosey goosey to precise and you've just learned something, even if the metric doesn't necessarily get better.

Yeah. And just to run through hunch and the rubric I use for measuring product market fit for the listeners, so H is for hair on fire value proposition, U is for usage being high, N is for a high NPS score, C is for churn being low, and H is for a high LTV to CAC ratio to capture the unit economics part of the equation. And, and I try to give an analytical view or lens on product market fit because founders really struggle with how to, how to determine this. Sean ellis has the 40% test, which is a very famous test that I also like to apply. And I sort of use that in the hair on fire value prop. It's embedded there because if someone were to take away your product, would more than 40% of your users be very disappointed? Which is what Sean's test reflects. And if that's true, then you know you have a hair on fire value proposition. A true must have. So to answer your question, Greg, yes, there are some analytical metrics and yes, there are some subjective metrics or nuance about whether you have true insight into the customer needs. And at Flybridge and other places in startup land, people refer to this as the earned secret. Do you really have an earned secret about a nuanced insight that maybe is non consensus, non obvious about what the need is of that customer? And as you continue to go down your learning curve, you begin to realize what those nuanced non obvious insights are and that's when you really see the magic.

Yeah, but I mean, do you have any ways of measuring, I don't know, the speed of learning the, the Quality of learning, the trajectory of learning.

I don't. That's a great question. I'm going to think more on that one because I really don't.

And so I mean there it's, I guess still somewhat qualitative. I mean it seems like venture capital, you cannot make venture capital into, I mean I always joke that it shouldn't even be considered part of the finance department. Right. Because in finance it's like you're measuring the value of a mortgage backed security down to the 8th dec. Decimal place. There's no room for that really in venture.

Yeah. And not only will I think more about the learning measurements, but I'll also reflect on how I do it in the classroom because obviously I am grading in essence, 90 entrepreneurial students as they go down their own learning curve and run their own experimentation machines inside their ideation process.

Yeah, I mean the other thing is, I think hard to understand is the degree to which you can rely on pattern recognition.

Right?

So you know, every algorithm has some bias and every algorithm has biased towards the past, right? Because you're, you're essentially looking at what's happened in the past. And a lot of people have said, well, why don't we just apply machine learning or predictive machine learning to the business of venture capital, right? You know, just examine all these different startups that are looking for funding and then run them through some model the same way Facebook, you know, runs, you know, post through some recommendation engine. And it seems like, I mean that's an area where we'll really never be able to do that because you're always looking for things that have never happened before. So how do you balance the, the sort of universal rules with the. Every case is unique when you're teaching venture capital.

And yeah, it's, it's, it goes back to that non determinism, non determinism as you said. And I would say that I like to share with founders the methodologies, the rubrics, the frameworks. But then I tell them, only you can apply this framework to your situation. You know, it the best. And it reminds me a little bit of when my now deceased, may he rest in peace, colleague Clay Christensen tells the story of going to Andy Grove of Intel and having Andy say to him, look here's the situation. We're debating the strategic issue with memory versus CPUs. What do you think we should do? And Clay would go off on a description of his case study about Rebar Steel and his disruptive innovation theory. And Andy would say, no, no, no, no, I want you to answer my question, what should we do? And Clay would go back to his Rebar Steel case study and he'd say, he'd say, andy, I can't answer your question. I can give you the case study and the theories and the frameworks, but only you know the answer to your situation and how to apply those frameworks to your particular case. So that's sort of the mindset I have as a, as a teacher is I'm going to present a series of case studies and a series of frameworks and a series of hard earned lessons from my own entrepreneurial days and my own investing days. But the students have to apply that to their startup and their situation.

I mean, it's a peculiar form of learning and it seems like venture and entrepreneurship is ideally suited for case method, perhaps in the way that other business decisions are not.

Right. You sound like you're giving an advertisement for Harvard Business School and for my course. But yes, I do think that is true.

Well, yeah, I mean it's kind of sad, but at the beginning of my classes that are case based, I referenced them to the Harvard Business School homepage and I say go watch all these videos on how the case method works because unfortunately that's the only place where you have this kind of instruction.

Right?

I mean, online, giving them a little bit of heads up about what to expect.

Yeah, exactly, exactly. It's actually one of the reasons I think the HBS online courses and I created an online course for my launching Tech Ventures class, which your listeners can take at HBS online to make it more available and more broadly accessible. Because I think this case method is really the best way to learn entrepreneurship.

Well, let's dig in maybe to how generative AI is changing what it means to be a founder right on the day to day. Because we've been talking mostly about the timeless methods. Maybe we can talk more about the timely tools you said. I think somewhere in the book, or maybe it was one of your posts on LinkedIn that your students said that they'd wish they'd learned this stuff earlier so they wouldn't have wasted so much time. So, so how can you use these tools to from day one, as a.

Founder, the first assignment I use in my class is I tell the students, go to your favorite tool of choice. Let's just use ChatGPT as an example. Treat it like a co founder and ideate with it and push it to come up with startup ideas. It's like you're having a conversation with the smartest person in the universe. Why wouldn't you take the time to test your ideas off of them and push them to help generate ideas for you? And the more information you can feed it, the more context. Your resume or LinkedIn profile, some of your work history, your HBS essays that articulate what you're passionate about, customer discovery interviews that maybe you've done, research reports that really interest you. The more you can feed it, the more salient those ideas are going to be. And you can use NotebookLM, Google's tool to throw in content and have dialogue with that's more contextualized. You can use ChatGPT or you can use actions, whatever you want to do and then ask it to help you design a few experiments to test some of the hypotheses to see if it's a really good idea or not. Push it. You know, is this value proposition really a must have hair on fire value prop? Is this customer segment, my icp, my ideal customer profile, a narrow enough definition? Push it, push it, push it and see where you can go with it and then begin to work your way through the process that I articulate in the book the methodology of customer value prop experiments go to market experiments, profit formula experiments. So for example, yesterday I was with one of my startups and the new VP of sales, they hired their, you know, their way down the learning curve. They hired their first VP of sales. They've moved from founder selling to now trying to build a repeatable sales model. And he said, what's your advice on how we can think about overcoming objections because we're struggling with getting customers to create a sense of urgency. And I said, well, do you record your sales calls? He said absolutely. I said well why don't you take all your sales call recordings and throw it into NotebookLM and put your website in there and your product description and your value proposition and then have a conversation about what are all these, you know, how do I overcome these objections? How do I create a sense of urgency? How do I build a sense of trust? Again, you're dealing with the smartest human, the smartest, not human, but the smartest agent on the planet. To be able to answer some of these questions, you just need to feed it the right context and then that continues all the way through your go to market, scaling your demos, your scripts, your qualification criteria, your marketing through customer service, building customer service agents and agentic capabilities to have 7 by 24 availability FAQs, all the way through your financial analysis and reporting auto generating board reports and investor memos. You know, all of those Things you can, you can instrument your entire workflow. But what I didn't get to Greg was the building part, which is using replit, codium, Cursor, Google Copilot to actually build out the applications. My partner Jesse was bending my ear yesterday about how he's been using Replit to build a new application just by chatting with it on his phone and watching it work. He says it's like if ChatGPT felt like the wizard Replit built on top of these new capabilities and agentic power feels like the wizard squared. I mean, it's just extraordinary what these tools are now allowing me to do.

Yeah, it's amazing because I've been giving talks now for the last couple of months talking about how it's the engineering team that you'll be able to scale much better. Right. They're the ones that are going to be more productive because of all the different coding tools that are out there. But I think what you're saying is it's not just the engineering team.

It's not. No, it's the product managing. The product management team can use Chat PRD to create product requirements documents and the solo founder can use Spark Rockets to build landing pages and test write out interview guides. I mean, the list goes on and on about what each function and each type of task that you're trying to achieve. There are tools available in addition to the baseline, foundational tools like ChatGPT and Gemini and what have you. There's these incredible special purpose tools that are bespoke we created for that particular functional task.

Yeah, I recently did a consulting project for a company and the project we spent five days together, the IT team for this company and myself and some other consultants. And the goal of the week long project was to figure out how this company could launch 100, 150/plus initiatives to integrate generative AI into every aspect of their business and ultimately to prioritize the first 10 applications. So we spent five days brainstorming and doing all this stuff, listening to vendors, et cetera. But as an experiment, we at the very beginning uploaded all the public documents of this company and the bios of all the people who are going to be in attendance. And we asked ChatGPT to just have a conversation about, you know, how to prioritize AI and to pick like 10, you know, easy wins that they could start off with. And, and we didn't even look at it until the end of the five days. And then we compared what ChatGPT recommended and what we had spent five days and many thousands of dollars doing. And it was actually really, really good. It was pretty close. Yeah. And so wondering, like, what, what value did we exactly add that you couldn't get for $20 a month because it didn't even use any proprietary corporate information. So, you know, obviously the role of the consultant's going to have to change. The role of the founder is changing. What about the role of the venture capitalist? I mean, do you see the job of the venture capitalist also scaling in this way?

It is in subtle ways. We are using the tools, like many of our peers, to be better at identifying signals and looking for founders and companies that might be gaining momentum or doing something new in areas that we find exciting. And then automating some of our outreach and CRM like capabilities with those founders. We're using it for our reporting purposes to streamline the ingestion of financial information and the outputs that we need to provide to our investors. And also just to brief each other. You know, I just recently have been doing due diligence on a company and loaded up all the due diligence notes and research reports into a notebook LM and made that available to my partners so that they can have a chat with all the insights that I might be able to provide them with before we walk into the partners meeting on Monday to discuss the investment. So there's a. I think a huge number of things that we're seeing that we and others like us are going to be using these tools to be more effective and efficient.

So do you think that this is going to make being a founder more accessible? I mean, that you'll see more people saying, oh yeah, I can give this thing a go.

I hope so. I hope what it provides, and Sam Altman talks about this in his blog post around New Year's, I hope it provides this explosion of creativity because people now realize that by having these AI co founders and thought partners and sounding boards, they can take some of these risks and come up with some of these ideas themselves. And also because of these building tools being so easy that they can become builders that they don't need to wait to hire an engineer or convince themselves not to even pursue it because they don't have an engineering background or they're afraid to work with an outsourced development team, they can actually do the work themselves. So, yeah, I do think it's going to have, and I'm really looking forward to this. I'm already seeing it, this explosion of creativity and building in our society.

And so then in terms of the people that you want to recruit to your founding team. Are they going to be different types of people? I mean, we already talked about they don't need to be machine learning PhDs, but what skills do they need to have?

I opened the book with this line that AI is not going to replace founders anytime soon, but founders who use AI are going to replace founders who don't. And I also believe that joiners who use AI are going to replace joiners who don't. That our portfolio companies are looking for AI native employees. And we may see a world where employees, candidates come to companies instead of with a team of engineers or marketers or salespeople as we have saw seen in the past as the HubSpot mafia travels from company to company. And in my case, the open market mafia was traveling from company to company, the PayPal mafia, obviously, very famously. Instead we're going to see employees travel with AI agents that they're going to come and say, look, I've got this team of AI agents that I've been building for the last few years to do my work in my previous company and I can bring that now to your company to be 10x more effective and efficient. So I think there's going to be just a really rich set of opportunities for AI native joiners and there's going to be a high bar that will be tested by employers about whether they're, whether individuals are AI native and facile with the AI tools.

So these 10x founders or 100x founders, are they going to be different than the 10x founders pregen AI?

I mean, I do think so, yeah. I think, look, the experimentation mindset, which I obviously the book's called the experimentation machine, that having that experimentation mindset and a willingness to try stuff and an ability to pick up new tools and try them out, that's going to be a really critical new capability and not just be a great leader of people, but a great leader of capabilities and tools and agents. Frankly, you know, we're going to need to see organizations that are great at agentic management.

Yeah, but in a way, I mean, if experimentation is cheaper, right? So when experiments were expensive, then you know, you really did have to be very judicious in terms of which experiments you ran. So prioritizing those experiments was an essential skill that you had to have. But if, if experimentation is cheap, then can't you just like spray and pray and say, all right, well, I'm just going to, you know, if I want to find my ideal customer profile, I just spray out, you know, 50,000 experiments and you know, that's how I discover it. So I mean, does that mean that you have to have an experimental mindset, but you don't really have to be particularly good at identifying, you know, how to target those experiments because it's so cheap?

I do believe judgment and strategy will still have timeless value and creativity. That said, if you look at some of the best experimentation, companies take booking.com, who runs thousands of experiments a day to a B test different wording, value propositions, pricing, positioning on web pages, and they're always, always, always optimizing. You're going to see companies instrument their entire company in that way. I do believe. And at the same time, companies are going to have to make strategic decisions about how to build those capabilities and how to apply judgment to decide on what are the criteria that matter the most, what are the true north metrics? When is it right to just allow your instincts and hunches to make a decision versus when do you want to be really data driven? One of the things I, I don't talk about in the book, but I speak of a lot in my class is that tension between hunch driven and data driven and something that I had Fred Wilson from USV in my class a number of years ago and he made this observation which I found very compelling, is that in the early days, founders should be more hunch driven and in the later days, post product market fit more data driven. And I think the notion that founders need to leverage their strategic thinking and creativity and human judgment and apply that again and again to prioritize these scarce resources. Even if the resources are, can be stretched more fully, it's still a competitive market and everybody is stretching the resources and being more productive. I still think that judgment is going to be very valuable.

Well, last question. You know, you've been teaching the business school for a long time and you've been teaching these courses on launching new ventures and venture capital and so forth. And I guess I'm, you know, I would always wonder the extent to which it makes sense to have these as separate courses or whether or not, you know, we should just be teaching this to everybody, right? I mean, you know, there's a debate over ethics. Should we have a separate ethics class or should it be baked into every class? Should entrepreneurship and these things you're talking about the experimental mindset and understanding how to sequence one's queries and so forth. I mean, shouldn't every business school student be exposed to this? I mean, isn't every company now basically trying to become a day one company.

I think that's super insightful. Greg and I have been pleasantly surprised, but I guess I shouldn't be at how executives at larger companies have reacted to some of these themes that I'm talking about. My main audience are founders and joiners and entrepreneurs. But every company is trying to be more innovative. Every company, every nonprofit, every government agency. I have a colleague at hbs, Professor Mitch Weiss, who's at the intersection of government and civic infrastructure and entrepreneurship, and he talks about the one employee, one mayor, City Hall. You don't need various staff and various departments because the mayor can use AI to ask the right questions and get the right data and make the right decisions and write the right memos and laws and. And ordinances. So I think that these capabilities, this mindset, yes, I think it's going to be universal.

Now, if you're like me, I think you probably had your students ask you how you can apply this to careers. Right? So everyone now is a founder and CEO of their own individual life trajectory, Right. And so concepts like strategy and so forth can be applied to their lives. Do you see these tools helping people to negotiate their lives better? I mean, you know, just going through life is a series of experiments you're trying out. Do I. Should I be a founder? Should I go work? Should I go work at General Motors? Should I get married? You know, where should I live? I mean, all of those are part of the process of learning. Do you see people sitting down and asking chatgpt, you know, like, who they should get married to and so forth? I mean, is that.

I don't want to. I don't want to go into the world of AI girlfriends and boyfriends and partners. But I will say that this mindset is definitely a mindset that people are applying. I've seen this across, as I said, 2500 students over 15 years. The entrepreneurial mindset, the experimentation mindset, people are applying it in very productive ways to live better, richer, more wholesome lives.

Well, Jeff, thanks so much for joining me. The book here, I only have the free print. It's called the Experimentation Machine Finding Product Market Fit in the Age of AI. And indeed, it talks about timeless methods and timely tools.

Check it out. Thanks, Greg. Great to see you. Unsiloed podcast is produced by University fm. Elevating the stories of your institution.