#207 Data Driven Venture Capital with Andre Retterath, Partner at Earlybird VC

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming.

It can be to keep up with.

The rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated.

The best learning content to help you.

And your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team.

We make learning practical, engaging and flexible.

It just works. Power your data and AI transformation journey on datacamp.com for us, it's very important to scrutinize the team to fully understand if they can and are willing to go the whole journey to build this multibillion dollar plus company that's number one.

Welcome to Data Frame. This is Richie Investing is something that can be incredibly data driven on Data Framed. We've previously talked about algorithmic investing where humans simply set the trading rules and let computers do all the work of buying and selling. One area of investing that has so far been resistant to making much use of data is venture capital. Particularly with early stage venture capital where VCs invest in brand new startups, there's no data yet on the company's performance and so the investment approach is traditionally largely about relationships and gut feeling. The idea that you can't use data just isn't true. There are plenty of existing companies that used to be brand new startups that you can use for comparison. That means that as with every other industry, the idea of data driven decision making is gaining ground. Today's guest is someone who's been spearheading the cultural change in venture capital to promote the use of data driven investing. Andre Reterath is a partner at Early Bird Venture Capital. His investment focus is productivity tools for data and AI, and he's been instrumental in creating tools to help his company become more data driven. Andre also runs the Data Driven VC side that contains news and advice about Data Driven venture Capital as well as tools for implementing the advice. I'm keen to discover how data is used in the investment process and how other industries can benefit from these techniques and become more data driven themselves. Hi Andrej, thank you for joining me.

Hi Richie, thanks for having me.

Just to begin with, what exactly is Data Driven Venture capital?

It actually started from me writing back then all of my research. I started writing more regularly on medium and then I just wanted to give it a name at some point so I translated it into a substack newsletter and it was called Data Driven vc Because it was really about leveraging data and AI in the venture capital industry. And today it has grown beyond just the newsletter. It's a community. We have different kind of streams from the first online conference to different kind of reports and so on. So it's really a community and quite an ecosystem around that movement.

That's very cool. But I think maybe coming from the world of data, it seems fairly obvious that you ought to use data in order to invest in things. So what? Why doesn't every VC make a lot of use of data?

That's the same question I asked probably seven years ago. I come off from a engineering data science kind of background, and when I randomly stumbled into vc, I was really surprised how manual and inefficient all of the workflows were. So I couldn't believe back then that those people who backed the most visionary, disruptive founders in the world still work themselves, like in the 1950s, or mostly the same. I think one digitization step that happened in the industry is really from pen and paper to PowerPoint and Excel, but that's really it. It was obviously data. Some people are like, yeah, why do we call it data driven? Of course we use data to inform our decision. But when I think about data driven, it's really leveraging modern data technology, modern data stack, and then also AI, more intelligent models and algorithms to first of all collect the data and then process the data to eventually get to a better decision, to be cautious of biases, to make sure you have all of the information available to make an informed decision, and so on. And when I first got into venture in around 2017, it was mostly manual. It's people sitting there. So one example is, I don't know, if you go to conference, you download the conference list, you have 400 participants or something, and then you start looking them up. You start with Crunchbase, DRAM, PitchBook, you look them up on LinkedIn, you look for news mentions and stuff like that. And this manual research process of just collecting the data takes days and weeks and the same is then true. Once you have the data available at hand, then you need to somehow structure it, put it into your internal format, you need to create a memo, and then also you need to balance these data points based on your previous experience. So. So that is where all of the different kind of biases like recency bias, similarity bias, confirmation bias and stuff like that comes into play, which is partially important for investment decision making, but oftentimes also neglects the more widely available sample of information. So Just because you had this specific experience with this one founder that might have been bad does not mean that this is to be generalized as a bad experience with a founder. And with these more comprehensive data driven approaches, you can suddenly consider a wider spectrum of data. And that was just not the case. Like when I got into venture, there were a few funds, really a handful of funds that have tried it when big data was a thing. And they stopped because people said, number one, it's all private data. So venture capital is a subsector of private equity. So it's private investors investing into private companies. And then by nature, this data is not as available as for public companies. Number two, we invest mostly in early stage. That means data is more qualitative versus quantitative. If you invest in public companies or growth scale up companies, the data, specifically the traction data, commercial data, is more quantitative by nature. And those were the arguments that people had and said, look, data driven VC as an approach and early stage venture will just not work.

Yeah, that certainly makes sense that if you're investing in an early stage company, there's less data available compared to a publicly traded company that's been around for decades. So I'd love to get into some of the details on where data is used, but maybe just at a high level. You mentioned you can use data for researching companies or founders, but can you talk me through just what are the different areas where you do actually use data?

Sure. I think to start with what's most important is to really understand the process. So for those who are not too familiar with venture capital, we first need to collect money from someone. This is what we call the limited partner, the LPs. So we have our own fundraising process where we collect the data. Once we have the money available, we go into the investment process. And the investment process you can dissect into different stages. The first one is sourcing. That means we need to build up the funnel of all opportunities and we need to enrich this funnel. So we need to collect as much information as possible about every single company in the funnel. And we have the screening, which is the second stage, means we need to go from tens and hundreds of thousands of opportunities top of funnel, and we need to narrow it down to a number of opportunities that's manageable by humans. This was where different kind of screening algorithms, but then also specifically the enrichment data, that can be time series data, that can be quantitative, qualitative data of whatever nature will be used and processed in different models to attribute a likelihood of success to these specific companies. So the Idea is really to say this company is more interesting than this company. And this is how we narrow down the funnel of opportunities. Once we have a number that we can manage as humans, we get into the due diligence process. So we meet the founder, we do proper due diligence on a specific company, we create competitive landscapes, we benchmark the metrics and all of this stuff. So this is what happens in the due diligence, where we also do customer references, we do references on soundwear and so on. And once we have all of this data available, we actually prepare it in an investment memo. And this investment memo is what we use as a baseline to inform our wider team and the investment committee to make positive investment decision. And then after the investment decision, it flips actually around because so far we've been diligencing on the founders. But then we need to convince the founders to actually do potentially competitive deals with us, like, we are the right partner for you here. And then once we have this transaction closed, this is where the fourth part starts. This is portfolio value creation. So we do introductions to customers, to potential hires, we do introductions to follow on investors, for follow on financing rounds and so on. And then hopefully if things go well, after whatever, six, seven, eight years, we will have an exit and then we can return the money to our investors. So this is the process sourcing, screening, due diligence and closing, portfolio value creation and exit. And along these lines is how we structured our modern data stack. So for the sourcing, it's all about building up the funnel. So we need to collect data. We connect data I dissected into primary and secondary data. Secondary data is established data aggregators. So companies that act as an intermediary and collect different information from whatever providers. So this is the classical CrunchBase, Droom, PitchBook, CB Insights, VentureSource Traction and so on, you name it. And I've benchmarked them in a study before in terms of coverage and accuracy. And we are just about to publish an update on this. The second dimension is then the primary or verticalized data. So this is for example LinkedIn data. This is for example buyer intent data. This is for example customer references news data. So very specific data about all of the companies. And then we merge this primary and secondary data together. For the secondary data, you can just buy it so you can integrate it via an API for the primary data providers, they oftentimes don't have an API. So this is what we crawl mostly ourselves. So we have built very scalable web scrapers that all legally, if we need to buy a license. We buy a license. We use different kind of proxy networks to emulate human behavior, collect this data and then merge it together with the secondary data. And then in addition to primary and secondary data, we also have our network data. So we integrate into our email and calendar to get this knowledge graph of different kind of nodes in our network. And then it's really about entity matching, merging all of that together and creating a single source of truth. So this is how we start top of funnel. And then obviously it's more about algorithms for screening and making the data actionable.

That's absolutely fascinating. And also seems like if you're one of these primary data source providers and you don't have an API so other people can make use of it, you're probably missing a trick there. Okay, so mentioned, like there's these four different stages. You up there sourcing, screening, due diligence and then the value creation. If you're getting started, like which area should you target? Like where is data going to add the most value to begin with?

That's a great question and I think there are different approaches to it. One, let's say a bit more academic top down approach is you just look at the value creation. There are different studies showing that in early stage venture capital majority of the value and more specifically studies say about 2/3 of the value is created in the sourcing and screening stages of the investment process. Which means venture capital is a finding and picking the winners game. So this is different than venture capital where by the way also you have typically a power law distribution in returns. So if we invest, we do a portfolio as an early stage investor of 30, 35 investments, we have this power law distribution. And for those who don't know what that means, you might know Pareto principle, the 8020 principle. And early stage investing is similar to the 8020 principle but with an higher alpha coefficient means we rather have like 10% of the portfolio that deliver 90% of the returns. And that being said, it's completely different to later stage investors or private equity investors that have more of normal distribution in their returns. So if we look at it, for us it's this power law distribution in the returns. And for us we start with getting comprehensive coverage top of funnel and then narrowing down. And this power law is specifically important because for us the worst thing that can happen is not to invest in a company that we need to write off that gets into insolvency whatsoever. That's part of the business. The worst thing that can happen is that you are in front of an outlier, but you misjudgment and you deselect them. So this is really the false negatives that we need to reduce. So and data science speed we need to improve for the recoil rate. And this is why most funds actually based on this value driven approach, start now with the sourcing and screening process and work their way through the value chain to due diligence and then also into the portfolio value creation and then beyond the stack into the back office.

Okay, so if I understood this correctly, because you're dealing with early stage startups, most of these startups you invest in are going to fail, you get nothing and then some are going to get an enormous 100 times thousand times return. Is that, is that the idea?

Yeah, I think like, like our, our idea is really to multiply our funds several times so you get into the top quartile if you deliver around 20% net IRR. And if you do the math, the typical venture fund runs 10 years. So if you do 10 years without compounding, you just take 20%, that's 10 times 20%. So it's 200% and you need to return the initial money, so that's 300% means you need to three times the fund to get into the top quartile of performance. And obviously our ambition is higher than that. So we aim for like five, four, six times every single fund that we invest in. And from this essentially we have this power law distribution. I said 30 to 35 investments, so it's not the majority as you might assume, but probably a third of the investment properly written off. So it's zero. And then you have around half of the investments that collectively return once the fund. And then you probably have another handful of investments that return more than like 3, 4, 5x but then you hopefully have whatever 2 or 3 that are the major outliers in the fund that return 50, 100 times the initial investment and that drive the returns for the fund to make the difference between, between the 2x fund and the 6,7 8x fund.

Okay, so when you got this huge difference in outcomes, it sounds like the selection process can be incredibly important. Can you talk me through what that involves?

Totally. So if we look at companies and like first of all I like to describe the manual processes and then how we map that into our data infrastructure. So if we look at the team at the early stage ventures, the most important one is always the team. So if we do postmortems like why did companies not take off? It's in most Cases, something related to the team. So the CEO or the management didn't grow into their new responsibilities. There were conflicts among the founders and so on and so forth. And there's tons of research on that available. And that's the most important because it's something that yes, you can rely on external credentials, like which university did they go to, other serial entrepreneurs, did they work for tier one employer before? And so on and so forth. So these are the external credentials. But what's super important is to sit together with the founders, look each other in the eyes and really understand like do they have the passion, are they going for the big outcome that we all need. Like for us as venture investors, because of this power law distribution, every single company needs to have the potential to become a multibillion dollar company, otherwise we would not invest. And for us it's very important to scrutinize the team to fully understand if they can and are willing to go the whole journey to build this multibillion dollar plus company. That's number one. Number two is then obviously related to the market. So we need to look into the market and understand is it even possible to build a multi billion dollar company if you take conservative multiples today? So let's say on an enterprise software company, let's say to take it easy, it's a 10x on the revenues. And if you want to build a 5 billion company, company needs to make 500 million in revenue, which means at, let's say an aggressive market share of 10%, the market needs to be at least 5 billion or something. So we need to understand what is the absolute market size, what are relative market dynamics, what is the distribution in the market, are there incumbents, is it white versus brownfield and so on. So those are all market related question marks. Then we look into the product and the technology. So to really understand is this different, is this differentiated from the others? And we spend tons of time diligencing that. It's also what I enjoy most, like really looking into the product and technology with the founders to really see what they have built or are about to build, given that we also invest at the very earliest stages. And from that you can also see the question towards competition, like how is this different than XYZ is already doing? So we spend significant time thinking about competitive landscape, how it's different, how are others funded, what is your new approach? And from that we get into another bucket that we call defensibility. On defensibility, we split it up into backward looking defensibility, which means once you have signed a customer and are working with your customer, how difficult is it for this specific customer to switch away? So this means also switching costs, can you increase switching costs for your customers? And then the second one is a forward looking locking effect. And that's everything from economies of scale, for example, network effect, data network effects and stuff like that, that with every additional customer it makes it even easier to win the next customer. And you can build up these very strong locking effects that prevent others from catching up. And in the data sphere, that means specifically you have access to unique data that help you to train better models. With better models you can attract new customers. With new customers, you can collect more data. And that's a data flywheel. So that's something we look into. And then obviously if the company is already a bit more mature, we look into traction, commercial traction, metrics, absolute, relative and so on and so forth. So those are on a higher level, the different kind of buckets of dimensions that we look into when assessing the company. And now it's about translating that all into the data stack. How can we collect the data to automate parts of this assessment or automate parts of the data collection for this specific assessment?

Okay, so last one lies there. And it seems like some of these things like understanding the total addressable market or like the other market dynamics like defensibility, it seems that those ought to be sort of straightforward to do in a quantitative way. Other stuff like you mentioned the team and the founders, how do you assess that in, in a quantitative way? Like how do you do data on this person's really committed to building a multi billion dollar company?

Yeah, that's a great question. And that's, that's also what we tried to prove people wrong when they initially said it's not possible for qualitative data. To give you an example. So let's assume there are two founders and we scraped the LinkedIn profile of the CEO. So from the LinkedIn profile, which is initially one data source, we can extract her previous academic experience, we can extract the previous work experience, we can extract interaction with other polls like liking, we can look at the sentiment and so on. Now we extract these features and then we process these features. So let's look into the academic experience. From the academic experience, we can look into the highest degree, then we can look at the university and the subject. So we have built dictionaries to represent university rankings. So all of the university rankings that are out there, both from which kind of papers within a specific domain and so on. But then also the subject because there are some universities which might classify as a tier 2, but if you study a specific subject at this university, that's actually considered as tier one. So it's a combination of the subject and university that helps us with like hard coded dictionaries that are dynamically changing over time obviously, but helps us to assess the academic credentials. Then from the highest year of your first degree, we know with a statistical significance your age, so we can imply your age. Let's continue the same logic for your previous work experience. So let's assume you worked at Google. You would typically say that's amazing. But if you, let's say worked at front desk Google for two years, that might not be as useful for an enterprise software company versus if you have done full stack development for six years at Google. So you can look at the duration someone spent at a specific position, you can look at the progress across seniorities, you can look at the combination of position at a specific, specific company that implies also expertise. Then you can also look at previous founding experience, you can look at previous founding experience with the same founding team or different founding team. And then you can also look the third dimension into their social interactions. So which stuff did they like? There are some research streams that we combined, one that is around Cambridge Analytica, so you can actually take the social media interaction from people and statistically significant predict the big five character traits. That's one research stream. And that means if you like, I don't know, Adidas for example, you're more of an introvert. But if you like Nike, you're more of an extrovert. Simplify it. And we take this research stream and couple it together with another research stream in marriage research, that's from a U.S. professor. And in this, you know, with the big five character traits of a couple, you can predict with a statistical significance if this couple will stay together. So we merged these different kind of research streams to predict if there's a likelihood for founder conflicts and so on. Just to give you a feeling beyond is this an introvert or extrovert, how active are they? And so on and so forth. And I just wanted to describe this example in a bit more detail to give you a feeling of what we can actually extract just from one data source being LinkedIn. And there's tons more like we create hundreds of features just from LinkedIn that is by nature all qualitative, but can be semi quantified and then leveraged in our algorithms for prioritization of opportunities.

I shouldn't be surprised that it actually takes hundreds and hundreds of different features just to quantify a single person because people are quite complex talking about the big five character traits. So this things like extroversion and how conscientious you are and how open you are to new ideas. Are there any of those character traits you found when you make for good leaders or make for good interactions between leaders?

Yep. And I have published some of that research through Data Driven VC through my newsletter before and the answer is it depends. So we have something that in statistical world we call the moderator effect. So we have the dependent and the independent variables. Let's say the variables we are looking at are I don't know, the team and so on and let's say the market and then the business model and actually the market and business model can act as a moderator effect for which team characteristics are required to build a successful company. More specifically, that means if we want to train a model to identify E commerce founders, like we have tons of data about successful E commerce founders. We will most likely find that those are people that are stronger in execution and excellence, what we call it. So most likely people that spend time at, I don't know, McKinsey, BCG, Bain Consulting, IB, Morgan Stanley, JP Morgan and so on, studied at a business university, did an mba. So more execution heavy people and they are more likely to be successful in an E commerce business just by the nature of the business. Now if we look at the moderator effect and change it to let's say core fusion, if we look into core fusion then suddenly people that have PhD professors, postdocs, like very research heavy have a higher likelihood to be successful in that space than someone, let's say with the consulting and business background. So we see this moderator effect and that's strongest for market and business model combination that determine the impact of the input variables like the team, like the competitive landscape and so on on the success of the company.

Okay, that's absolutely fascinating that different industries require sort of different personality types and different experience. Maybe that. Now I said that out loud. It should, it should be slightly obvious.

But it's good that there's one, there's one common ground. Actually there's one common ground and we call it the entrepreneurial compass. And this is very difficult to quantify. This is really something of like are these people too willing to hustle through everything? What' the intrinsic motivation, are they driven by money? Are they driven by intrinsics? Is it something like back in their childhood they need to prove their father wrong and stuff like that? This is very difficult to project from the outside. But those are the very intrinsic motivations. And we work our way through collecting more data, more innovative approaches to predict as much as possible of that. But eventually it always requires, in my perspective, human setting together, spending time together and then making decision. Like we actually want to build this together. And this is why I personally believe like we have the traditional world of venture capital where everything is done manual and you meet your friends at the whatever golf house and you make a deal and it's like we will invest in this company. This is like the world that we've seen probably 50 years ago and still see today for majority of the industry, to be frank. And then we have this new movement of data driven VCS that leverage data and then the data driven VCs among themselves. Can we split into the camp of augmented VCs? So those are VCs like myself, like Early Bird, that we believe in a combination of data driven approaches and human in the loop. We call it an augmented VC approach because eventually it's always human making decision. We believe that founders do not want an algorithm on their board. We believe they want to work with humans. If shit, that's the fan, they want to call someone. And then there is the camp of full quant funds. So there are funds, very few of them, it's very early. But they say you can even do early stage venture without any human involved. Like the human is the root cause of all problems of wrong investment decision. And we remove the human from the equation. So we have the traditional and then the data driven world. And within the data driven world you have the augmented VCs and the pure quant that are just very few and starting out. And we see ourselves as a very early mover in this augmented camp.

So you've got data to assist your decision in the augmented camp, but then you're as a human, you're sort of making that final decision 100%. So it's interesting that you mentioned the quant approach saying like humans are responsible for all the bad decisions. And you mentioned some biases before. Can you talk me through a bit more on like what are these biases that might make you make wrong investment decisions?

It's a super interesting topic which funny enough I also wrote about. I wrote a dedicated article about these biases among investors and if I recall correctly it's a bit more than 20 biases, I think 25 or something where investors are most prone and it's like similarity bias. Or the founder went to the same university and like I invest in someone from Alma mater similar like that. It can be something like confirmation bias. So you have a thesis on the market and then the founder tells you this thesis and you feel like, oh, like this founder is amazing, he's just confirming or she's just confirming what you had in mind already. Or you have something like recency bias where you've worked with a founder that has done something fantastically right and then you're looking for exactly the same kind of characteristics and new founders and so on. And that might be good. Like some of the greatest investors, they probably have an amazing gut feeling. So as a venture capital investor, it's not only about analytics. I think that's the hygiene factor. Everyone needs to be analytically strong. But there are also people who have operational experience that have different experience and just a great gut feeling for people, they just know they can read people in a way of like is this a great entrepreneur? And if you have this gut feeling that's very difficult to replicate and scale up. So this is also the people that strongly believe in. It's, it's a college industry, it's all like handcraft and you cannot scale this up. But the problem is that all of these biases are created through a very limited sample. So if you look around as an early stage investor I mentioned, we have portfolio of 30, 35 companies that we build up over three to four years. So say as a partnership, we do, let's say ballpark, 1012 something investments per year. Obviously we add on that, follow on fundings and so on. But let's say over a time period of five years, every partner probably does 10 investments. As an early stage investor and if you look at the success and failure rates that I mentioned before, over the course of five years, if you do a good job, you probably see one or two like really successful companies, some mediocre and some you will write off. And then if you project that even further, like venture is a long term business. So even though the company might raise follow on funding and we see early indication of success, oftentimes it takes 6, 7, 8, 9, 10 years or more to actually return money. So for us as early stage venture capital investors, it takes a decade to know if we are any good. Like you can hit one, that can be luck, that can be whatever. But you need to replicate it, you need to do it twice or three times. And the problem is that until you get to this point like 15, 20 years out, your biases will be shaped by a very small sample size and you strongly believe that these biases are right. Now, with data driven approaches, you can rely on the global sample of all companies that ever became successful and you can look at mutual patterns that these successful companies had in common. And for me, it's a combination of both. I think the pure data driven approach in my perspective also doesn't work because eventually you need to sit down with the founder and you need to see, you need to trust your gut. Like, does it feel right? Do I want to work with this person for the next five to 10 years? And also the other side is for me just looking at a limited part of the sample, which can be the right one. If you work with the right founders and you have the right biases, but in most cases you miss some part of the wider reality.

Okay, so it just seemed like if you got someone who's really, really good at reading people, they're probably going with their gut more actually. Certainly I've met people like that and they are incredibly good most of the time, but then when they get it wrong, they're like, they won't change or won't listen to the data because they believe they're right all the time. Okay, so let's talk about how you make this happen at your own company. So if you want to make your company more data driven, who do you need to hire?

It's a great question. So just for context, Early Bird here in Europe is one of the most established, one of the oldest venture capital firms. So the firm has been around since 1997. So it's 27 years in the market and went through different kind of ups and downs. That means we have also a legacy, positive and negative, and there is an established culture. So when I came in in 2017 for the first time, so I joined in parallel to my PhD then full time, early 2018, and the established culture was already very analytical and technical in a way. That's also why I joined Early Bird in the first place, because it was not like your traditional consumer investor or so was very technical in nature already and a strong belief in more structured processes. So for me it was comparably easy to also say and convince the other partners that we would need to leverage more data driven approaches. So in the first place, I took my PhD research and built an MVP myself. So across the data collection, the entity matching, how you represent the data in a knowledge graph and build a first simple MVP version. And in 2020 I presented it to my partners and started said, look, for me it's very clear that this is the future of venture capital and we need to Go this direction. And to be frank, it was an easy sell. And that was also a learning that I had from other VC firms. If you don't have buy in from the general partner, so the general partners are the ones that own the firm. Eventually if you do additional investments in whatever direction it goes out of their pockets. And if you don't have this buy in, you can still stop it right away. So you need full buy in from the budget owners. And from there onwards I decided to hire first a generalist software developer and then I had many learnings along the way of how to build a team from I hired two deep expertise, I hired two generalists and so on. So there was a wide spectrum of learnings. But today we have a team constantly around eight people which we call our engineering team within Early Bird. And this team is autonomously building our platform that we call Eagle Eye. So it's the sourcing and screening platform that everyone in the investment team uses. And that has also helped us already to identify a few very promising investments.

Okay, so I like that you've got a mix of people who are experts in one particular area and people who are generalists and hopefully between that you got some sort of, you got the right level of coverage in all the areas that you need. So you mentioned the importance of getting managers on board in order to get the right culture for having this data driven approach. Do you have any advice on how to get that management buy in and how to get that cultural change to encourage more use of data?

Yeah, I think one is just like in terms of values and how you think about the world in the first place. I think that needs to be aligned no matter what. If this is off, if people believe it is all just handcraft and I don't care about data, I know it better than the data. If this is the mindset of the people, then you don't even need to start finding arguments. If you have people who are receptive to these arguments, then provide them with the arguments. And this is also what I tried with data driven VC like to provide as many arguments quantitative qualitative nature for us internally to actually get going. So what I did back then is also because one argument was from people, okay, let's assume we can collect this data. Let's assume we can represent these companies and find their digital footprints. But how can we trust with the most important part of our investment process for a machine? And I mentioned before 2 3rd of the value is created in sourcing and screening. And screening is really how to narrow down the funnel. And then you have this power law distribution. If you weed out the wrong company, you might really miss this outlier company. So people struggle to trust this selection process. And what I did is I trained different kind of machine learning model. Back then I worked mostly with an XG boost classification model to classify companies into successful and unsuccessful. And what we did is actually we did a dedicated study to benchmarking humans against the machine, which is super interesting. And essentially we took input data from, I think back then, it was 2015, about a sample of companies. We followed these companies until 2019 or 20, and then we looked at which of them became successful by investor standards and which failed. So we classified them into 1 being all the success cases and 0 being all of the failures. And then we looked at the input data from 2015 to predict the successful cases as of 2020. And with that, we've built a different kind of classification models. Now we took this model and we provided it with information from companies that the model has not seen before. We took the same information, we anonymized it, put it into one pagers, and presented it to investors like human investors. And we always ask them, do you recognize this company because it's true companies? And then if people said yes, they were deselected for the study. And if they said no, we asked them, out of 10 companies, which are the five that you think will be successful and which are the five that you think will be a failure? And the split in the sample was, in reality it was five, five. And we could find that even the best investor, we had around 110 or 120 investors. And it's like it includes all of the top names, all of the funds participated in that. And even the best investor in the sample missed one of the successful opportunities. And our machine learning model also had a recall of 80%, meaning the model also missed one of the successful opportunities. The average, however, for the human was around 60% and the median was also in this ballpark. So what we could show is that the model performs at least as good as the best human investor that we had in our sample. And this model does that on a recurring basis. And it performs significantly better than the average or median investor out there. So what we can say is with these screening models, even the most untrained investors will be capable of identifying promising opportunities if you enable them with these models. And that was something. When we did this study, I presented it to my partners and everyone was like, okay, if this is really true, then it's a no brainer like why would we look through tens of thousands of opportunities? And the only way to scale it up back then was to just hire more interns. So we had more deal flow. We moved from 8,000 to 10,000 to 12,000 per year. And the only way to end it is like okay, one intern can handle whatever thousand opportunities a year, so we just hire for more interns. And this one here is way more scalable and also trustworthy. And doing studies like that, like very tangible, helped me to convince the partners, get the full buy in and then also from the team to really roll it out within the team with top down support from all of us as a partners.

That's very cool that the models are getting the same performance as a top investor. Also think it's interesting that the performance of this model is kind of, or at least the cost is measured in like interns per year equivalent because that's.

The only way to really scale it up. Like there are two options. One is to not reply to this. And we distinguish between inbound means founders reaching out to us and that was the majority historically and outbound. We doing desk research using data driven the projects to proactively reach out to the founders. And we found that majority of this inbound is actually not a great fit because founders don't know about our investment criteria and so on and so forth and. And the only way to scale it up is to hire more people. So I remember back then we had also at times significantly more interns because we wanted to handle the whole volume and we wanted to get back to every single entrepreneur that reached out to us.

Just getting into some of the techniques you're using. You've talked a lot about using traditional sort of machine learning models for doing the screening like trying to predict success and then the sourcing. It was a lot of like web scraping, making use of APIs. I'm curious if there are any other techniques you're using. And well, I mean generative AI has been the big thing in the last year or two. So I'm wondering whether you're making use of that anywhere in the process.

100%. Also as full disclosure, we are thankful to be one of the very early investors in a company called Aleph Alpha, which here in Europe is one of the leading companies in the gen AI space. So they mostly serve enterprises and clients with very sensitive data and we consider ourselves as one. So we have started playing around with large language models in 2021 already and integrating it into our systems to give you a feeling on how we Use it. I'm happy, happy to share some use cases. So number one is within our team we also have sector expertise. So me personally, I'm looking into enterprise software, AI developer tools, data tools and so on. And we have like a huge matrix of responsibilities within our team. And we started actually in 2018 to manually label all of these opportunities. So we have manually labeled more than 50,000 companies into this different kind of sectors. And then we did build a manual classification model back then which did not properly work to be frank. And today you can use large language models and you can classify at like whatever 98% accuracy to different kind of industries. So we use large language models to classify the industries, we classify the business model and so on just based on the semantic description of the company. Then another use case is we vectorize the description. So we also scrape the company websites, public register registrations and all of this stuff. Stuff. We vectorize that together. And then with these vector embeddings we look for similar companies so we can automate what I described earlier, the competitive landscape mapping, which back then was days and weeks of research going through all of these databases looking for similar companies like from left and right, trying different angles to find them. Today we can do that by a click of a button. So for all of the companies in our system, we have already automatically created a competitive landscape based on the vector embeddings of the descriptions of the respective companies. So we always know live what these companies look like. And then we have also the enrichment data like who are the investors, what's the funding, what is the news mentions, who are clients, what is the head count and all of this stuff. Another dimension that I can talk about is really the interaction with founders. So within our system Eagle Eye, we have trained on our historic email and other data different kind of LLMs. So we can essentially just by click of a button, generate an outreach email to the founder. Again for us it's very important to have human in the loop here because the worst thing that can happen as a system that randomly reach reaches out to people and burns our brand. So we decided proactively to keep the human in the loop. And we have several other use cases where we really use LLMs across the.

Value chain that first use it case in particular. I mean it maybe sounds a little bit pedestrian like using generative AI to clean your data, but I can imagine that's absolutely a huge time saver and cost saver, just improving your data quality and not having that. Well, I guess interns again like clicking on like this, this company's in this industry totally.

And I think one, one that is really one of the core of our infrastructure is what we call entity matching. Because if you just take this example, there is a founder who, I don't know, put something in Reddit forum and it's like I'm looking for a co founder or whatever kind of wording. This founder will get an entry in our database. Now let's say a week later she decides to put on her LinkedIn, working on something new or something fancy. So she will get a second entry, then she will go to the public register, register the company, she will get a third entry. Then they might launch something on GitHub, like a new project that will get a fourth entry, and so on and so forth. So suddenly we have like tens of different entries for the same company and we need to merge them together. So this is the deduplication and entity matching. And for that we also use some modern models where we can look at the vectors and compare similarity across these companies to do both inter and intra entity matching. So also within sources and across sources.

Okay, yeah. So again it's just making sure that your data is in good form and that seems like an incredibly important, important use case. Okay, so do you have any big success stories where maybe your models have found some company that was missed otherwise or you've otherwise had a big win?

So a few and I'm happy, happy to share that. But I think generally for us it's about identifying the right opportunities as early as possible. And I think timing as an early stage investor is incredibly important. Like you don't want to see an opportunity once it's obvious to everyone because then it might be incredibly expensive or competitive to be get into this deal. So for us it's really identifying these companies as early as possible. This is also why we look into forums, we look into stealth founders. We look like we follow thousands of stealth founders across Europe. If we have seen a digital footprint that they are about to start something new, if they think about starting something. So we start at the earliest point and some great examples is for example, I mentioned Aleph Alpha. Before Aleph Alpha we scraped when the company got registered. The CEO is a serial entrepreneur. He sold his previous company to Apple. From there onwards he was in the valley, he saw a lot about GPT, two different kind of transformer models, got back to Europe and then started this company with his incredibly strong technical co founders Amwe and the two of them started this company. Initially, very few information available. We led the series A round then in summer 2021 and we identified the company initially through their public register registration. Then they raised a seed round with some local investors that had a previous relationship and once they launched their website and the round was also public, we had sufficient data points to prioritize the opportunity and then through our network got in touch with them and actually six or seven months after their first round we then led their Series A round. So that's one example where most likely we would have seen them significantly later. Another example is a company called Ethan AI. It's an ETH spinoff in Zurich. We scrapped a company initially when it just got registered. So it's two founders who did their PhD in different areas at ETH Zurich. And we looked at the precedent round which back then was a bit too early, and then co led the seed round together with our friends at LA Familiar now General Catalyst. So that's another example and there are a few, few others. But we see some early evidence that it's actually delivering great deal flow earlier than the majority of the market can identify them.

Okay, that's pretty cool stuff. Although I guess you mentioned like you sort of need to wait 10 years just to get the true results.

It's too early to tell. It's too early today. So we have good initial evidence. Aleph Alpha, for example, they just raised a 500 million Series B round in November last year. So good early evidence. But as we know for early stage investors it's too early to tell. It only counts when you return money to your LPs.

All right, super. We'll bring you back in like 2034.

And you can give us an update and we have a few more examples to talk about.

Absolutely. So it seems like you spent quite a few years building up this capability. So can you talk me through, like are there any important lessons you've learned along the way? Like what mistakes have you made that other people should avoid?

I think what I mentioned earlier really the buy in from different kind of stakeholders, but then specifically the people who are on the budget. I've seen many funds and people within these funds complaining that the budget owners just do it for window dressing. And there was really a shift in the market when I started this whole stuff like seven years ago there were very few people in the market doing it and majority just said oh, it's not impossible because X, Y and Z. I mentioned two reasons before and there was a shift, I'd say probably around the time of ChatGPT where suddenly everyone is like ah, productivity, efficiency and so on and now everyone is telling more or less the same story. So there's also like a completely new industry that I call investment tech evolving. And in this investment tech we have players that do some of the components that we do internally, also externally and provide it to other VCs so they buy it off the shelf. Then our LPs are like, yeah guys, it all sounds the same of what you're doing, but in reality it's a huge, huge difference. And majority of the VCs today get pressure from the LPs from the market that they need to do something. And for them it's like yeah, we have this one, data engineer, data scientist, product manager, whatever. And it's for them really window dressing. And I know from several of these people within the organizations if you don't have the full buy in, it just sucks. It really sucks. And this is the Most important number one. Number two, I'd say learning is that 8020 applies here. Like don't over engineer it like we Germans oftentimes do. You don't need a perfect version, just get going. And I think one point to start off is really automations or simple automations, just freeing up time and then it can go more complex over time. And I think the third one that comes to mind is really on the hiring part. I would rather hire an Engineer with like 3, 4, 5 years of experience with a bit of a Swiss army knife versus someone with a PhD in NLP and 10 years of postdoc experience. Because what we need is people who can execute and build products really fast. And it's also speaking on eye level with the investment team because that's what you collectively need. The top down buy in from the management who are on the budgets. Stakeholder management. You need to deliver something really fast is 80 20. And you need to have the right engineers that really understand of what the investment team is doing because otherwise. And that's the ultimate blocker, it's culture. You only have one chance to try it out with the investment team. If they don't like it and you have delivered a premature version, they will be never changing their investment workflows. And that's the biggest struggle that I can tell from. Thankfully not too much my own experience, but what I heard from many others who have tried.

Okay, it sounds like these are actually quite related ideas. So if you hire someone who can build something quick and provide some value fast, that's going to help you get that executive buy in. That's then going to help you scale your program.

Yeah, it's a bit of a checking act. I think I bridged this checking act back then with with some research and stuff that I've built myself next to my job. And I think if you need to do and solve the cold start problem right now, probably start with with exactly this generalist who build something really fast, try it out for like 612 months, see if it delivers the results that you expect and then double down.

Excellent. All right, do you have any final advice for anyone who wants to become more data driven or help their company become more data driven?

I think too many advices to put them into into this conversation, but I think the easiest one is just to check out datadrivenbc IO. This is where I write about lots of this stuff. I'm pretty transparent on it. So I am a strong believer in open source in community. This is why I'm sharing majority of this stuff. Obviously we keep our secret sauce. So the stuff I'm writing about is probably something that we thought about a couple of years back. But it is great to start these conversations with the community and I've benefited a lot from this community so also want to give back there and together make this industry really more efficient, effective and also inclusive.

Wonderful. It's great that you're sharing all your content and your ideas. So yeah, I would say to our listeners, please do check out the data driven VC side. Excellent. All right, thank you for your time, Andre.

Thank you so much, Richie.