Episode 83: Startup Scrappiness, Venture Matchmaking, and Thinking In Bets with Leigh-Marie Braswell

Regardless of where you stand on the matter of data science sexiness, it's simply impossible to ignore the continuing importance of data as well as our ability to analyze, organize, and contextualize them. Data are changing how our businesses and societies function. They are being used to solve a ton of interesting problems and shaping the questions we can ask of the world around us. Hi, I'm James Lee, and this is DataCast. Join me for raw conversations with practitioners from the worlds of AI, machine learning, statistics, and data science.

Hi listeners, this is DataCast, where I hold long-form, in-depth conversations with data practitioners and researchers to unpack the My guest today is Leigh Marie Braswell, an investor at Founders Fund. Before joining Founders Fund, she was an early engineer and the first product manager at Scale AI, where she originally built and later led product development for the LiDAR and 3D annotation products used by many autonomous vehicles, robots, and AR/VR companies as a core step in the machine learning lifecycle. She also has done software development at Blend, machine learning at Google, and quantitative trading at Jane Street. Leigh Marie is originally from Alabama, graduated from MIT, and loves to play poker, run long distance, and scuba dive. It's my pleasure to have you on the show.

It's great to be here. Thanks so much for having me.

You know, just to start our conversation, so while doing the homework for our chat today, I found out that you were originally from Alabama, and then you went to Phillips Executive Academy where you were high school valedictorian. I believe you're also very into dancing ballet growing up and then later switched interest into solving math problems competitively. So yeah, would you mind sort of sharing some of these formative experiences, your upbringing, and how did your time at the best boarding school in the US and your passion for mathematics shape the person you have become today?

Yeah, so I grew up in a really small town in Alabama, very different from kind of my current surroundings, but basically When I was growing up, dance was something that I was— that everyone was just kind of pushed into. And I mean, there's definitely a lot of great lessons that you can learn from, you know, kind of committing to a hobby at an early age. But ultimately what happened was when I kind of went into the local middle school, they had a math team and I tried that out and I just discovered my love of solving really challenging problems. So it was the first time where I saw problems that really tested my critical thinking abilities, made me be very creative. And I just became so addicted to doing more and more of that. So kind of went down that rabbit hole, started to go into math camps outside of the state and then just went all in. So I saw that Exeter, the boarding school, was the best place to go in the US for competitive math. And so I actually came up with a pitch for my parents. So like PowerPoint slides, Of, you know, why I should go, why it was dangerous for me to be driving to this magnet school, just any reason that I could use to convince them. And thankfully, I was lucky enough that they were extremely supportive and, you know, were able to send me there for the last 2 years of high school.

Absolutely. What are some of the things that you learned from attending Exeter? I believe this is a very competitive environment where kids were being basically trained to, you know, join Ivy League later on. So are some of the things that you learned from those 2 years in boarding school?

It was really a head start in terms of kind of the experiences that people have in college. So just getting a giant course catalog. I did advanced economics classes, I did quantum mechanics classes. I was around a bunch of people that were a lot smarter than me. So it was just a great experience. It pushed me intellectually and then also showed me A lot more diversity of thought than I had had growing up.

Yeah. Having that head start really kind of setting up you to cultivate a lot of the passion later on throughout your college and, you know, right now professional life as well. And so for college, you went to MIT to study math and computer science. So kind of reflecting on those years in college, you know, how would you describe your overall academic experience at MIT? For instance, like what were some of your favorite math and CS courses taken?

Quite frankly, I will say one thing about going to Exeter and then MIT, at least at MIT, when you start the course load, I got a lot of it from Exeter. When I started, I actually had a little bit of kind of spare time on my hands. And I think that's what led me to eventually getting really into a lot of clubs. So I was big on the poker club, a business club, and then also just optimizing for diversity of internship experience and really going down a bunch of rabbit holes with my internships. My favorite classes at MIT definitely were, you know, later on in my course load. So a lot of algorithms classes, and then I started taking a few machine learning classes. And that was really when it clicked and I realized this is, this is the magic of this kind of marriage of like math and practical industry applications, and this is going to be the future. And so that's ultimately a large part of what led me into wanting to work with or work on machine learning after college.

And in general, how would you describe sort of the academic culture at MIT in terms of the focus on either practical experience or research industry to kind of hear that part of view?

Just kind of like, is there a research focus or is it more of a practical industry-focused school?

Yeah, in terms of course load.

Yeah, I mean, it's definitely very, very theoretical, and I think that's— it's definitely very interesting coming in as someone who had you know, did a little bit of research in high school and the beginning of college. It's very easy for you then to get kind of swept up into the research and academia track. But I think it was just those sorts of like practical internship experiences that I had that made me realize how interesting industry can be as well. And it's actually kind of a, what you would call a narrative violation. MIT is sometimes painted as this place where you go and everyone's very theoretical and very academic. But a lot of the best, for example, B2B enterprise unicorns in the past few years have had an MIT founder. So there is a strong contingent of people that are technical, but also very enterprising.

That's a very interesting perspective that you provided in terms of that culture. As you mentioned, you know, you really want to impart a variety of practical experience to compensate for the other half of the equation in terms of In addition to your theoretical knowledge. And so you took part in multiple internships throughout your undergrad. And, you know, based on my research, you spent a summer as a quant trader at Jane Street and then another summer as a software engineer at Blend. You know, kind of looking back at the internship experience, what were some of the technical challenges that you observed when working on, you know, building trading tools and banking products at those companies?

Yeah, so a wide variety of things. So I think at Jane Street, really the kind of most important lesson that I learned that I still use a lot to this day is really quantifying risk and being okay with taking risk as long as there's positive expected value or EV. So from day one at Jane Street, we would come in and the older traders would literally offer us maybe like 1.1 to 1 odds on a coin flip. We were expected to to always take that, right? It's positive EV. Like you can take the better side of that, of that bet, and you should be willing to put your money where your mouth is if you know that the odds are in your favor. And so that was kind of the sort of lesson that underpinned that whole internship. And then ultimately, you know, I take with me as a VC and as a poker player. And then in terms of what I learned most at Blend and at Jane Street, honestly, is when you're looking at the finance industry, there's just so many inefficiencies. So the mortgage application process is just so inefficient. And so, you know, to have someone like Glenn come in and streamline parts of the process, it just makes things work much better than they do. And it saves a lot of people time and money. And then at Jane Street, like I didn't realize how much of trading happens over the phone. Right. So I think even a few years ago, I was seeing like trading is hard even for institutions to do. And it's, you know, data is in a black box sometimes and it's the Access isn't democratized. So now we're seeing like all of these trading apps and infrastructure and a lot of this happening on the blockchain as well. And, you know, it just totally makes sense because right now financial institutions, I just kind of naively thought that they were extremely efficient and my internships exposed me to all the inefficiencies.

I see. And I'm just curious, looking a few more years ahead of time, you know, what are some of the areas of finance and trading besides those that you already mentioned can still Rife for disruption.

Yeah. I mean, one area that I've been seeing or hearing a lot about lately, I mean, I think there's still a ton you can do in the mortgage industry. Like there's still a lot of processes that happen that should be automated or basically automated. But if you're just looking at like finance, one area that's really interesting is just trading things that aren't public equities. Right. So why can't a normal person buy stake in a private company? Or any private company? Why can't we have, you know, different derivatives on, you know, private companies or events or whatever? I think like there's definitely a lot of players that are starting to have platforms here, but in terms of it's still not a super seamless experience or a super universal experience. So I think there's a lot you can do with democratizing access to both asset classes and then also information.

Awesome. Yeah. Thanks for sharing those perspectives. breaks up to kind of see how you set up an emotion and then solving those inconsistencies, as you mentioned. Circling back into your interest in machine learning, you mentioned a little bit earlier that you started taking some ML classes at MIT and became interested in sort of that domain. In particular, you also have worked at Google, first as an ML engineer on the research and machine intelligence team, and then later as an APM, associate product manager, for their Chrome web platform teams. What were some of the key insights about machine learning and product management that you observed from the experience of working at Google?

Yeah. So I think when I was at Google working on ML, I was working on basically NLP models to predict entity sentiment. And I think the main thing that I learned, well, both at Google and after my time at Google was how amazing the machine learning infrastructure at Google is. So, I mean, Google has tools to do everything, like optimizing hyperparameter tuning, you know, like just deploying a model, like being able to see when it goes off the rails, like retraining, all of that is so streamlined at Google. And it's not basically anywhere else or anywhere else that's not a big tech company. So I think the biggest lessons that I learned were, you know, just kind of how far, how different it is when I'm in my MIT class trying to train a model versus I'm at Google trying to train a model. That was one thing. And the second thing was even at Google, the training data process was probably like the most, one of the most painful parts of training a machine learning model. So like the training data that we had was sparse. It was hard to get new training data. Some of it was wrong. And, you know, that was just something that kind of followed, I guess, in the back of my mind. And when I started talking to Alex about scale, then it was something that I immediately had related pain points. When I was an ML engineer. So those were 2 big lessons. And then I guess if you're talking about product management at Google, I think there's no better place to kind of get the PM playbook. I mean, they essentially made it with the APM program because they were the first ones. So it definitely taught me all of the kind of important parts of being a PM when you're having to deal with multiple stakeholders and lots of processes and how important it is to Optimize processes.

So ML infrastructure, studied that at Google. The challenge is getting good label training data. And then, yeah, I learned a lot about product in that environment. Can you just kind of round up a little bit about your college experience? But, you know, you did quant trading, software, ML, and product. What is the unifying thread that combines them together? You know, when you're looking back at those different disciplines that you kind of tried to explore, right? in college? What's the thread that you keep pulling?

Yeah. I mean, I think the only thread that I can find is problem solving, right? Like, I've just— I love to go down rabbit holes and just learn a lot about things that I think are the future, like things that I think are very cool or interesting in some way. I mean, that's why I think it makes a lot of sense that I'm now in VC because There's no other job where you get, at least every week, you're almost expected to be learning about something completely different. But yeah, kind of trying out all those different experiences. I mean, I learned so much about how businesses are built, how to quantify risk, a lot of industries. And so I highly encourage anybody in college that has even the slightest amount of uncertainty around where they want to end up after college, just try a bunch of different things.

Yeah, I mean, that's a great answer as well. After graduating from MIT in 2017, now you decided to join— at the time, the company is called Scale API.

That was a good catch on your end. It was called Scale API.

And yeah, I believe it was less than 10 people at the time. And you joined initially as a full-stack engineer, right? Kind of like, you know, stepping back into that year, right? Like, What was the rationale behind your decision to onboard a very early-stage startup?

Yeah, that's a great— it's a great question because a lot of people at the time said I was crazy. So, I mean, definitely, as I said earlier, being able to point to my own understanding of the pain point that the company was solving, this need for training data for deep learning, and then also just kind of my desire, given my experiences in college and internships, to work more on machine learning and work with machine learning engineers. That was definitely going to be the end customer of Scale. But I mean, there, I guess there are a lot of companies I could have joined that I could have related to the pain points and, you know, gotten to work with ML engineers. But definitely can't understate the importance of Alex and the early team that he assembled. So I think I just knew that I would probably never have such faith in a team ever again. And then I would just have, you know, so much regret if I didn't join that team, because I had trust that Alex and the early engineers were really, really strong, not just on like an IQ level, but I think Alex probably has the highest combination IQ, you know, plus EQ of any person that I'd ever met then. And Probably now in the top few of people that I've met now. And so it was just such a special opportunity and I knew I had to join.

So a more big emphasis on sort of the people, right?

Yeah, no, I mean, of course the idea and all of that, but it would've been really tough to join this company that small if you don't really have faith in the people. And so I always advise friends that are thinking of joining very small startups, And CEOs or leadership of small startups, you got to spend a lot of time with the people that you're recruiting because, I mean, they're making a huge bet on your abilities.

I see. Yeah, for sure. And just on that point, do you think— this is a super general question, but do you think like more college grads should join early-stage startups? And if so, what's the framework for that? How to choose whether to do what you did or they can join even a later stage or, you know, What are some of your thoughts regarding advice for technically brilliant folks who want to make a dent in the industry?

Well, it'd definitely be a bit hypocritical of me to say don't go work at an early-stage startup. And I mean, to be completely honest, I think a lot of these high-potential technical people that you're talking about, I think a lot of times, and I'm guilty of this myself, the big tech companies or the more sort of stable opportunities will always be there. There's no time sensitivity or no urgency for you to join a company that hires hundreds of thousands of people every year if you're extremely technical and have the kind of skills to be able to pick whether or not I'm working at a startup or a big tech company. So if you find a startup that you really think is cool, I would suggest just going for it. You're going to learn so much regardless of how the startup turns out. And it's not like you're Limiting your opportunities down the line. You'll probably go into the next— if you have to go into another career, like you'll go in with so much more context on the real nuts and bolts of how things are built.

Yeah, I totally agree with that. And I guess like, you know, joining a startup at the time is like massive upside and limited downside, right? You're wearing multiple hats. Pick up a lot of relevant skills. Well, circling back into scale. So as they deeper into the technical problem that Scale was solving at the time. You know, Scale is a data platform to accelerate the development of AI, providing high-quality training data for data machine learning teams. Well, first of all, can you briefly explain why labeled data is the key bottleneck to the growth of the ML industry?

Absolutely. So I think the first thing that you have to believe, you know, training data is extremely important, is that deep learning is here to stay, or here, like, going to be here for a long time. So deep learning is essentially a form of ML that requires a ton of labeled training examples to work correctly. So think about it like a very common use case for deep learning right now is identifying objects and self-driving. And the way that deep learning works is for the deep learning algorithm to identify cars in all scenarios or in 99.9% of scenarios, they need to be given all these different edge cases as to what a car could look like. So cars, all makes, all models, all orientations, from different countries, in different lighting conditions. And so you need a ton of labeled training data to train your deep learning model to predict the cars. And you hear kind of every so often it comes up again and again, like, okay, well, this works, but it's extremely expensive. So There's a ton of research being done on different methods, like one-shot learning or few-shot learning is what the most common other methods are called. And supposedly you need a lot less training data with these methods, but to my knowledge, they still haven't really caught on in popularity. It's still very theoretical. And, you know, it's still, even the most sophisticated machine learning groups are using deep learning for their production models. So it's kind of training data is a necessary input into deep learning, which I think is necessary for a lot of these production machine learning use cases.

Definitely. And I think this has become even more important recently, just going from model-centric to data-centric approach to view applications. And I think, you know, Scale kind of ahead of the curve at the time. I think back in 2017, not a lot of people really pay attention to the importance of getting labeled data as being considered as a support unit almost rather than like a core function. So I think some of the work that your team is at scale building awareness about the importance of the problem. So I just want to say that to add on to your point. You were the first product manager at scale overseeing the 3D sensor fusion annotation product that supports a range of use cases from autonomous vehicles to robotics to augmented and virtual reality. What are some of the unique engineering and product challenges of dealing with 3D sensor data?

So there are definitely a lot of challenges to dealing with 3D sensor data. So if just kind of for a visualization of it in your head, if you've ever played The Sims and you're in like a 3D world, imagine if, you know, instead of the more realistic world, it was just a bunch of dots. So just dots with the XYZ position of where LiDAR reflected, and so therefore thinks that an object is or can sense that an object is. So there's a lot of challenges. It's hard to visualize. It's hard— there's no standard format for it. Like, most companies package or store their LiDAR data a bit differently, send it a bit differently. And it involves a moving car, so it's very easy for the data to not be calibrated correctly, and you get these weird artifacts in the data that you then have to figure out how you're going to label. So it's, yeah, it's pretty tricky. And then I guess even though it's tricky, it gave us the opportunity to create lots of standards ourselves. So we created, you know, or we at least talked with a lot of different customers to figure out, okay, here's the schema you're going to send it. Here's the interface that our labelers are going to use to navigate around in 3D. And then it also gave us the chance to, on our end, figure out how do we label this complex data as quickly as we can. So. Part of that was figuring out how to fix a lot of the artifacts that I was talking about before automatically, or doing other types of automation, whether it's pre-labeling some of the data with a machine learning model on our end. So before it goes to a human, figuring out ourselves based on past data where we think objects are. And that problem is pretty tricky. If you look at ways of labeling LiDAR data automatically, Like, do you use a bird's eye view of the data to train your ML model? Like, do you voxelize it in some way? There's still, you know, there's a lot of different methods and they have different degrees of accuracy and performance. And so we had to kind of figure out what worked for us.

Well, thanks for sharing such challenges. From your description, 3D data has become more and more universal in most of these cutting-edge use cases. So it makes sense to kind of tackle that. While doing the homework for this chat, again, You did a lot of great homework.

I really appreciate it. I feel like you know me already.

All right. Yeah. So, you know, try to be the best interviewer as possible I can. You know, you're definitely an interesting persona. But yeah, talk about that. I watched this talk that you gave back in 2019, basically talking about your experience building this distance-efficient annotation product from scratch, from gathering customer interest to building the initial MVP. Right. And just a couple of things that stood out, like you talked about the importance of speed and really like using it on scale a little bit. So yeah, can you unpack some of the key takeaways basically in that talk?

Yeah, absolutely. So it's definitely less of a technical talk. I think the intended audience was a bunch of product managers and aspiring product managers. So it was more so I wanted to give an overview of some of the things that I learned being a product manager at a startup, because I think being a product manager at a startup and being a product manager At different stages of a company's life, it's just very different. Your job changes all the time. So I think one of the key takeaways of the talk was, especially in the early days, so, you know, either pre-product market fit or very early product market fit for a startup, it is so important to listen to your customers. Because ultimately, if you're— especially if you're a B2B product, you're solving their problems. And hopefully they will pay you for solving those problems. So I think very early on, a lot of good product decisions that we made were because we had really good relationships with our customers and were able to build a general product that satisfied a lot of their pain points. And then another point in the talk was really just around continuously asking yourself at a startup, why not? Why can't we do this? And so at scale, sometimes it seemed almost impossible that we'd be able to provide this much labeled data with the current set of tooling and the current set of people. But we really pushed ourselves and always asked why not. And then more often than not, we figured out a way to make it work. And so then that goes back around to the problem-solving thread. When you're an early-stage startup, you really are a problem solver and you'll do anything to solve the problem. Some of my fondest memories of Scale are me, you know, making training videos for our taskers and things like that. So just wearing lots of different hats.

That makes a lot of sense. Customer empathy and big ambition, essentially what you're saying. I'm just curious, how does listening to customers actually look like in those days? Like, what are some of these, like, manual things that you have to do in order to accelerate the conversation-to-insight ratio?

Yeah, I mean, I think it's one, yeah, just kind of picking the customers that you think that having a really close partnership with them will create a product that ultimately you can sell widely. So just being very selective. And then like every day, if the customer is willing, asking for feedback, asking for tips, asking for how they're doing this internally. And then also, I mean, what's interesting about Scale is we don't just have the enterprise customer, we also have the human annotators that are working on doing the task. So we just very frequently, we would you know, should be sharing our screens with them, watching what they're doing, or having some sort of like recording and really just debugging like everything, every way that they interact with the tool. Because I mean, the best feedback is literally a video of what they're doing versus, you know, kind of the— as you allude to in another question, but the information compression of like having somebody do something and then tell you about them doing it. So we would go down to like customers' offices in South Bay all the time, and then we would fly to see our taskers all around the world. So ultimate, you know, one-on-one information.

Yeah, thanks for sharing that. It's like literally stepping into the office of your customers and show them, right, how that looks like on your screen.

Yeah, I've been coding in so many Uber— I thought, you know, before the pandemic, you know, you'd be coding in your Uber to the customer.

That's amazing. That's absolutely amazing.

Or writing your spec document or whatever. But yeah, no, unfortunately with COVID didn't do as much of that, but before COVID Yeah.

So we talked about like a phase of initially trying to find product-market fit, but over the years, this sensor-to-vision product has become more advanced with new features that support cuboids and segmentation tasks. And, you know, along that journey, the list of scales, enterprise customers also became more and more impressive. We got much more well-known brands. What have been some of your big learning curves to deal with these additional project over this scaling phase?

Yeah, that's a really great question. So one thing that I like to, I guess, remind people about Scale, because it can be really easy. I mean, because now that Scale is getting into so many different awesome things, especially like sort of ML, you know, SaaS products. Scale from day one was a very operational business. So it was all about how do you coordinate literally hundreds of thousands of people around the world? Do these tasks, whether it's a LIDAR labeling task or you read text and tag entities or something like that. And so that's kind of the crux of the problem. And it's, I think the insight that we had that was different than a lot of our competitors was you can't just give people the tools. The really hard part to build is the quality control system. And so each kind of milestone in our growth in terms of like volume of tasks that we were doing, that quality control system had to become more sophisticated. Otherwise, there would be quality problems, or we would not be able to get through the task. And so just figuring out how to automate confidence on each label or on each like unit of work that started out as very manual, like literally, we just have people that we trusted look at every task, sometimes those people would be us. And then, you know, as we scaled up and up, we do basic automated checks. And then at some point we got to, you know, ML checks, and then we're, you know, dynamically doing levels of review. Like maybe like if there's a beginner labeler, then the next person that sees it will be a more advanced labeler. And conversely, if you're pretty experienced, maybe we don't need somebody after you to see it. We just need our automated system. Right. So at each level of kind of scaling and each new logo that we got with maybe a slightly different use case, We have to make the system a lot more sophisticated. Building the labeling tools was the easy part.

When you got new customers, I suppose they may have different requests in terms of the type of data they want to get. How do you like— how to scale in general from that operational perspective that you mentioned, prioritize what requests make sense to put more focus on and which one is like less urgent, I suppose?

Got it. So the question is more around like prioritization once we started to get all these different requests.

Exactly.

Yeah. I mean, it's definitely not an easy question. I think the sort of most naive prioritization you could do is just, you know, total contract value, right? And just kind of blindly prioritize that way. But I think what a better prioritization for you to do is a bit of that, but then a bit of what can generalize really well and where do we feel like the industry is going to go? And that's how like both products that you brought up, 3D Cuboids, 3D segmentation. At the time, they weren't the thing that, like, if you had added up the total contract value that we knew about, they weren't the thing that was the biggest. Like, we knew that there was a lot more demand for 2D, or we thought that there was a lot more demand for 2D labeling when we built 3D. But we knew that by building it, we'd get a lot better data as to, like, the market size. And so we just suspected that that's where the industry was headed, and that's where a lot of people were spending money and then had some initial proof points, but then ultimately kind of still prioritized it anyway. And it was a great decision because it turns out that people were spending a ton of money on this and just doing it internally. So I think it's a little bit of current, since you're an enterprise business, a little bit of current data that you have, but then it's also, you need to do your research and develop an opinion about where the industry is going and how general these sorts of Like product features are going to be across different customers.

So just kind of related to that point, like, I don't know if you still keep in touch with computer vision applications in a variety of industries. Are you seeing like 3D data keep being utilized more widely? And if so, like, is it the only mode of working with data or would there be like even more advanced level of data that practitioners have to deal with?

Yeah. Okay. So, I mean, It's interesting because, I mean, every year a ton of new people start their machine learning journey. So, you know, even though the most advanced customers are now moving on to using like 2D and 3D and also maybe some other sensors, there will be customers that still get a ton of value out of just images, for instance, right? So like, I've seen so many different cool robotics companies pop up. in the past, you know, 12 months that are still getting a ton of value out of just image-based models. But then I look at the self-driving car customers, and they're starting to, you know, become more and more sophisticated with the types of data they're using. And then even in, you know, non-computer vision use cases, I'm also starting to see sensor fusion. So people doing really cool things combining audio and video and doing ML on the combination of those. So it's, it's definitely exciting. I think it's one of the industries where You know, tons of people still haven't even started really using ML in production. And so even basic machine learning will be helpful, but then the leaders are continuously innovating and figuring out new ways to make their models better with potentially different types of data.

Thanks for sharing that insight. Let's take off your product hat, put on your company building hat. Given your pivotal role taking Scale from 0 to 1, an unusual thing that I noticed about Scale is the company's relentless speed of execution. Why do you think that speed matters for scale? And in particular, like, how did the early team institute a focus on moving fast as the company expands?

So I think there are a few reasons why speed is really critical for scale in particular. So, I mean, as I've been saying, labeled data, it's a necessary input into a machine learning engineer's workflow. And so it's just super mission critical for them to get as much label data that's high quality, that covers lots of other edge cases as quickly as possible. So that definitely necessitated speed because that was an important thing our customers were measuring us on. And then secondarily, I think once people realized that scale was taking off and there was this giant need for training data, there were a ton of competitors that popped up. And when you have competitors, if you don't accelerate your progress, then it becomes a lot harder to be the sort of leading company in the space. So that was another reason why we needed to move very quickly. I think in terms of how did we sort of institute the focus on speed or really make the focus on speed part of the company, it was definitely a lot of the people that we hired. We just really screened for that quality in particular. So I think there's a blog post that Alex wrote about hiring people that really, really care about their work. And I think that that was something that we focused on very early on. And so you had a lot of very ambitious people join, a lot of ex-founders, a lot of people with startup experience. And then, you know, we would literally hire such a— or we still hire amazing people at Scale. The kind of combination of a ton of really scrappy people kind of pushes people to work harder and smarter. And I think that culture is very great. And then I think as scale grew much, much larger, then making that sort of the need for speed and iteration part of the company credos. So there would be like, we had a list of different credos that would get repeated a lot and be called out, like when somebody would exemplify one of them. And so I think there was a credo. That had something to do with iteration speed.

Thanks for sharing those, those 5, and especially I love your point about scrappiness. I think that's one of the qualities that's extremely underrated for a lot of people.

I agree. Yeah.

And yeah, you talk about like the importance of hiring scrappy folks and that's kind of lends itself pretty well to my next question. So hiring is another critical responsibility of any early stage startup. You know, as part of the early founding team, like what valuable lesson did you learn to attract the right people who are excited about Skillz's mission?

Yeah, I mean, definitely I would recommend reading Alex's blog post on this for sure. But the one thing that I'll emphasize is the importance of having a really good behavioral interview. So just having very good questions that will uncover signals that hopefully will be correlated with, with how people actually act once they join your company and start working. So I think in terms of the types of people that you want to select for, it's people that really take a lot of personal responsibility for their own decisions and their work, and people that just love learning. So love to go down lots of rabbit holes and are very naturally curious. But I think there was one thing that stood out to me when I was kind of jamming on potential behavioral interview questions with some people at Scale. But I think the single most important thing you want to select for is somebody that can stare down potential failure in the face, like probably likely failure in the face, and really believe that they can work and they can change it. Like they can make a failure a success because at a startup, you know, the cards are definitely stacked against you. And so you need people that believe that they can change how things are going to turn out.

And I'll be sure to include that blog post into the show notes so listeners can have a chance to take a look and dive deeper into some of the points that you just mentioned. And I'm just curious, besides the emphasis on behavioral interview and really kind of screen for those How to screen for, what is the early team have a good sort of network, personal network and professional network? And how do you leverage some of the network from maybe for you, like Google, MIT, and transfer and blend? What are some of the things that you do in order to spread the importance of labeled data and early-stage startups and things like that to some of your friends who might work at more prestigious companies and just scale?

Yeah. So you're asking like, how do you essentially like use your network to hire into a startup. Got it. Yeah. Now, I mean, I think we all realized early on that it's better to be shameless. So that basically means like everyone in your network should know that you work at Scale, and everybody should know like how well things are going and how exciting of a time it is to be working on those sorts of problems. So we would just constantly like, we would do talks at You know, different schools, we would post on social media all the time, write a lot of content. You know, I think I annoyed my friends all the time. I still annoy them as a VC, but I annoyed my friends all the time just with, okay, well, here's what we're doing at Scale. And I think you could be perfect for this role. And, you know, do you want to come in and talk to Alex and, you know, learn, learn more? So I think it's really just about like, even if you don't think somebody is looking, at least giving them the option. Or giving them more information about how they could potentially beat scale.

That's a great answer. So I came across this insightful memo that Alex sent to the Scale team back in 2019, which he made public. And a lot of people can take a look at that, but essentially he emphasized the importance of developing uncompressed understandings of how everything works together as scale grows. How does such a practice manifest itself into the day-to-day operations of the company?

Definitely, in summary, just we did a lot of things in person and we did a lot of, you know, meetings and jam sessions just like in person because that's the least compressed way of conveying information. I mean, so it's interesting because there's a trade-off though. I mean, obviously the more you do in person, the less you do over writing or something like that, and it can, in the negative case, like take up a lot of people's time. So it's definitely not like you should always do everything in person and there should be nothing compressed. And it's something that, you know, as a company grows, I think you have to be more cognizant of, because when you're 5 people or 10 people, it's totally fine to just get everybody in a room and talk about things. But yeah, not when you're 500 people. But it's interesting, because I think one of my biggest mistakes early on as a product manager was that I thought that compressing information and saving engineers time was something that I should optimize for. So I, you know, I would go to a customer meeting and take really good notes, and then give them to the engineers to say, okay, here's the context. And I don't want to like disturb any of your building time. Because when I was an engineer, you know, I know that if you kind of get in the flow and then somebody disturbs you, it can be pretty challenging to recover from that. But ultimately, like, I actually kind of reversed my stance on that when I realized that you do lose a lot of things unintentionally when you compress information. And so at least having engineers in some meetings directly with customers, directly with labelers, Actually makes you more efficient in the long run because people have a lot more context.

I see. In person and really get the right end users and the developers into the conversation so they have the right context into it. Absolutely. Kind of reflecting on your 4-year scale. I mean, at this point you left the company, but where do you see scale in the next decade? I believe like a lot of things that Alex has been giving in press and things like that, you definitely want to become the fundamental infrastructure, like the data infrastructure company. But yeah, I'm just curious to hear your thoughts as an insider, as someone who been part of that journey, what do you see Scale in the next 5 or 10 years?

Yeah, I mean, I'm definitely very excited to see what Scale does over the next few years and in the long term. I mean, the opportunity for them is just so massive. I think starting with being the place that you go for training data and then expanding outwards from that, especially if you just look at the rate at which the amount of money being spent on machine learning is growing. If you look at the different types of people that are now involved in machine learning, there's just so much that you can do providing them tools and helping them with their data. So I definitely think there's just so many cool— and I'm sure you've read many awesome new use cases, whether it's Scale Nucleus, where you can upload all of your data and see where your dataset isn't balanced and things like that, or whether it's different labeling types, or whether it's even something different than that. I'm sure they have lots and lots of things in the works. I'm just so excited to see them become one of the machine learning companies that's really powering our movement to ML, because it's definitely still very early.

For sure excited to see the Station company and how it grows in the future. Looking back at your career a little bit. So I believe that around 2020, you have started to angel invest in early-stage enterprise companies with technical founders. So what advice would you give for aspiring investors who contemplate their initial few angel investments?

If you're getting started with angel investing, I definitely recommend using kind of the simple heuristic of investing in awesome people that you trust. So, I mean, more often than not, when you first get involved in angel investing, you're investing in your close friends, which is kind of what got me into it. And then also, ideally, you're making an investment in an area that you kind of understand well, or you really want to learn more about. So, you know, when I saw a close friend of mine doing something with machine learning, at that point I was like, okay, you can kind of diligence this yourself and you really trust this person and you'd love to work with them. So why not angel invest? And I think once I started doing that and just kind of realizing how awesome it was to get to learn about different types of companies, And how fun it was going through those early stage days with my friends again, and hopefully trying to be like one of the higher value to money investors on the cap table. It was definitely just kind of a super fun experience. And what happens when you start providing value is you like, your friends will refer you to more companies. And then you'll also be like, you'll see some of the startups that they use or that Scale uses, and then you can reach out to them, or maybe you develop a thesis from like your day-to-day work. and just start sourcing companies from there. And so then you can get more and more sort of involved in the ecosystem and develop a brand for yourself. And I kind of treated it, I only did it for about a year, but as a learning experience. So I think there's a really good book on angel investing that Jason Calacanis wrote. And he quips that you can either spend money on an MBA or you can angel invest. And I'm not sure if that's necessarily the best idea for everybody, but I definitely can kind of see the value in it if you really are very principled about your angel investing and learning from your angel investments. It's a great learning opportunity.

Yeah, absolutely. And I'll be sure to include that in the show notes so listeners can have a chance to take a look and read that book. And Jason's definitely a great name in the field. I suppose during that year you honed your muscle of investing, right? Since March of this year, you have been an associate investor at Tharos Fund. A firm with an incredible track record that also, I believe, invested in the Series C round of Skillz, right? What about Founders Fund's investment manifesto that attracted you to join the firm?

So I've always admired Founders Fund's track record. Like, you look at some of the companies that they've done— Facebook very early, Stripe, SpaceX, Palantir— it's really been incredible, just kind of the bets that they've made on the future and how they've turned out. And then secondarily, I really love their general ethos. And I think this is clear in the manifesto of being brave and questioning the status quo, even when it's not the popular move. And yeah, just being very, very honest. So, I mean, definitely both of those things. And I knew that ultimately joining Founders Fund, I'd get the chance to partner with companies that are really passionate about building a better future and not afraid to make a crazy bet. on themselves, like a lot of the other companies in their portfolio. So definitely, like, all of that, reading through the manifesto resonated really well with me. And just talking to the Founders Fund team, everybody is insanely smart. So it pushes me to be a lot deeper of a thinker.

Yeah, that seems like you found the ideal environment. Again, like, kind of back to the early part when, you know, choosing a new venture is really about the team and the people you associate with. And it seems like You filed that first one, right? Just like a side note about venture in general, I think like a lot of people have, especially practitioners, technical people, have different interpretations and understanding of like, you know, the venture industry. Now, as someone who like just joined a VC firm, like what do you think is the biggest misconception that people have about being a VC?

Hmm, the biggest misconception. I think it depends on the type of person that you are, what your biggest misconception is. But it's definitely, I guess the one that I got the most was, oh, well, like, you know, at least your day will be more relaxing than being at a startup or something like that. And to that, I'd say it's definitely a misconception. I think the best VCs, which I mean, definitely I'm not, but I'm trying to, aspiring to be, work extremely hard. So it's just a job where there's always a company you could be meeting. There's always something else you could be learning. There's always, you know, Being more responsive, like being more proactive. It's tough. And figuring out your own strategy of like how, you know, you'll best partner with companies and how best you'll find those companies and diligence and then ultimately partner with them. It's really challenging. So I mean, the biggest misconception I hear is that it's like more of a retirement job.

Yeah, that's great. Yeah, like cultivating stamina and optimizing for long-term partnership. It sounds like kind of the thesis that you just mentioned. That kind of lends itself pretty well to my next question. So out of curiosity, as a new associate investor at the firm, how have you been proving your value upfront, showing the value that you provide in some of the venture deals at Founders Fund thus far, more specifically in some of the deals with co-investors or with any people you want to collaborate with in the future?

So when I, when I joined Founders Fund, definitely like By necessity of me being the newest team member, I'm the most fresh out of operating. So I've been doing a lot of sort of providing a lot of operator perspective on different sorts of deals. So, you know, talking to customers, doing a lot of product diligence for anything, even from early stage, which is more of what I focus on, to even later stage, providing that sort of technical perspective. So definitely that's kind of been a way that I've been showing value from day one. And then I guess secondarily, like Really growing and utilizing my network. So not only the Scale Network, but from all my internships and from school, and then finding sort of— and I recommend this for any investor, but, you know, once you start to figure out, okay, I look at a lot of, for example, dev tools, data tools, and ML infrastructure companies. I now have sort of friends that are angels or practitioners themselves, and we'll very frequently talk about those sorts of areas. So I find a company and think that it's pretty good, I'll like talk about it with, you know, 5 people in my network that are the kind of closest to that area. So that's been super helpful, just kind of having these sort of very close-knit sort of diligence groups. And then obviously they like to talk to the companies too. And then, yeah, I mean, that, that those are kind of been the main things growing the network, obviously. So happy that you asked me to be on the podcast. So growing brand and then, yeah, just developing different theses. So for example, you know, I've been doing a lot of work in the data infrastructure space and probably going to publish a blog post on some of that soon. So I'm excited to— maybe, maybe if you're interested, you can read it.

Yeah, for sure. And I'll definitely include that in show notes when, when it's being released so people can observe and contemplate some of the ideas that Leigh Marie is going to provide. I guess the main thing you mentioned is crowdsourced insights from friends in order to cultivate investment thesis, right, which is definitely important for early-stage investors. This is another thing I'm curious, like, you know, given your background in operation, working at Scale in the past, like, what advice have you been giving to some of the portfolio companies in terms of, in general, like, hiring decisions and expanding the founding teams? Because I'm sure there's definitely a lot of startups who are undergoing the same challenges that Scale used to have back in the days.

Yeah, that's a great question. So Definitely on the hiring side. I mean, one, I always try to help through my own network. Just have a lot of friends who are trying to, you know, get into startups or change jobs or whatever. I'm always happy to make matches. And I think that's where investors can be really helpful is, you know, finding people to work at the startups that they work with. And then just providing general advice. So I think it's a really important question. When do I hire my first product manager? And you know, what kind of product manager do I want them to be? I think there's a— depending on what type of company you are, depending on how your team is composed, the answer to that is pretty different. And then, yeah, like, how do I— if they're an enterprise business, I can provide insight into, you know, how should I price my product and how should, like, we set up a sales team, especially if it's a technical product. So kind of helping out with advice there. I mean, I'm always happy to share my scale experiences and Definitely got a ton of them, you know, seeing Scale grow over 4 years.

Absolutely. I think your experience with Scale really gives you that credibility that a lot of technical founders can resonate with as they ask this question. Reflecting on the arc of your professional identities thus far, what have been some of the surprising lessons that you found during this transition from software engineering to product management and now to venture capital?

Yeah. So I guess to summarize it, You know, from engineering to PM, it's you know how do you build a business? I think when you're an engineer, it's how do you build a product? I think when you're product manager, you're working with most arms of the business to build a business. So whether it's sales or you know marketing or design or biz ops or data, like as a product manager, you're working with all of them to accomplish overall. KPIs. So that transition, and then from PM to VC, it's how do you evaluate all types of businesses? And then eventually, you know, how do you, how do you grow all types of businesses as you partner with more and more companies? So it's even like a more zoomed out version of that. So I think kind of the unifying thread might be like in all 3 jobs to get really good at any of them, it's all about you kind of finding your superpower. Whether that's a certain type of engineering or a different investment strategy, but ultimately you kind of master the basics and then you figure out what you're really good at and lean into that.

Yeah. And if I'm correct, I was listening to one of our other podcast appearances, but the idea is like, as you transition throughout this phase, like your hindsight, your perspective going from super specific, zooming in, and now step out, zooming out a little bit, because as an investor, it's important to have High-level understanding of everything and then evaluate from that principle. Whereas, you know, as an engineer, you focus on nitty-gritty details. So it seems like that's the way you have transitioned your career as well. Get a broader scope of responsibility, both from personal growth and as well as from even your network as well. I want to round up our main conversation on a fun note. So one of your hobbies is to play poker. What have you learned from playing poker that benefits your careers in startup and venture?

Great question. So I basically summarize it by saying, you know, every decision that you make is a bet on the future. So there's some probability of it working out. There's some probability of it not working out, but the goal is over time to make enough good bets such that your decisions will be right in sort of the fullness of time. So, I mean, this happens in poker, like Jane Street taught me, and I should be more disciplined about this now, but A lot of people would have spreadsheets and every time they played a poker game, they would like, you know, put out how much money they'd won or how much they had lost that you could look back over your last 20 games and see like, okay, you know, yes, I'm making good decisions or no, I'm not making good decisions. Because ultimately, like, you could be a great poker player, but have a really bad game because you got very unlucky, even though you played it correctly. But over many days or many games, you know, you'll be able to see your positive expected value. So that's kind of how you should think of your career too.

For sure. It's like decision-making under uncertainty, being focused on the decision and detached from the outcome. I was just reading a couple of books recently from professional poker players, some of the one-on-one. I thought that Annie Duke and Maria Konnikova, those kind of books talk about some of these principles, and it seems like you learn a lot from them and apply that to your career as well. Fascinating insight. So, Leigh Marie, do you play poker yourself? Well, not yet. I'm just trying to read about it. If there's like an organized poker group, I'll join.

You should, you should, you should. I don't know if you're ever around the Bay Area, but would love to invite you to a poker game.

Yeah, absolutely. And we'll talk about this off the record. With this part of our conversation, I want to move on to the final closing segment in which I'm going to ask you 3 rapid-fire questions, and then you can give quick answers for the listeners. Number 1, name 3 people in the venture community whose work you admire.

So no explanations, just their names?

Well, I think it'd be great if you give a little bit of an explanation.

Okay. So Peter Thiel. I mean, he wrote the Bible kind of on all things investing, Zero to One. And I think a lot of his even crazier sort of bets on the future have been right. And then Ali Partovi. So he's founded a fund called NEO. He has sort of a general investment thesis of investing in very strong technical founders. Like very frequently he has them take programming tests. And he has a community around which he builds like a lot of college students that are very strong technically and shows them different startups. I think that's a really, really cool model of what he does. And then I guess the third person would be the person that I worked with pretty closely at Founders Fund. So Trey Stephens. He's just an extremely mission-driven investor and leader. So he's early Palantir, he co-founded Anduril, and he's a partner at Founders Fund. And it's just incredible. He is so very passionate about advancing all things government technology that he'll spend significant portions of his day working with all govtech companies, even ones that we aren't invested in, because he's just very driven to make sure that democracy is preserved and we're constantly on the forefront. So those would be my 3.

Thanks for sharing that. I'll be sure to include those profiles in the show notes so people can check it out. Number 2, name one book that you would recommend for people to cultivate better foresight.

Besides Zero to One, I definitely love the book Seven Powers: The Foundations of Business Strategy. So it's basically the book on what is a moat and what are all the possible moats of a business. So Whether it's like a network effect or switching cost or branding, it goes into deep detail about why each of these are very important and why to have a lasting business, you probably need multiple of them.

Just curious, what's your favorite power in that book?

I mean, I think my favorite part is just like, I literally have the 7 powers memorized. And when I see a company, I'll go through my head and say, okay, well, which of these does this company exhibit and why? And if I can't figure out one, Then I really need to kind of rethink, you know, like, how is this company actually not going to have 10 competitors down the line that do the same thing?

I add that to my reading list as well.

A good one.

And then lastly, imagine that you can send out a single tweet to all the early-stage VCs on Twitter. What would you tweet about?

I would ask them, what is the coolest company they are working with? Tell it to me.

Yeah.

Yeah, that would be my tweet. Show me the one coolest company you're working with.

I think that's a great way to end a conversation. I do see that, you know, Leigh Marie kind of playing the part of facilitating a lot of good discussion on Twitter and asking her network for deal flow and all that kind of things. So yeah, I'll be sure to include that in show notes so they can connect and follow and provide value to her work as well. So yeah, Leigh Marie, I really enjoyed chatting with you today, learning about Your original background for getting into mathematics to studying computer science at MIT, your multiple internship experience at Jane Street, then in Google, your decision to join Scale and help build the company from 0 to 1, the state of label training data and it's important for development and application, your transition into current role at Founders Fund making strategic venture investments for technical founders. And overall, this is a great conversation to kind of learn about And I think the unifying thread is like, you definitely focus on problem solving and really, really like, you know, that level of rigor and scrappiness, like, really take you to various ways. I'll be excited to see more, some of your upcoming investments in the future and hopefully see more startup founders benefit from the insight and Georgia perspective. So yeah, I really enjoyed it and I hope you have a great rest of your day.

Thanks so much. This was such a good conversation and excited to Well, that's a wrap for another episode of DataCast.

Hopefully you have learned something insightful and interesting from my guest today. You can read the show notes from the podcast website at datacast.simplecast.fm. If you want to get instant updates when a new episode is released, either follow me on Twitter, Or subscribe to my newsletter on my website jameskaley.com. It is my greatest pleasure that you listen to this podcast and take advantage of the data revolution coming upon us. Goodbye for now.

Thank you.