ETL Speaker Series: Alexandr Wang, Scale AI

Our speaker. Hello everyone, my name is Emily Ma and I'd love to welcome you to the Entrepreneurial Thought Leader series today presented by the Stanford Technology Ventures program which is the entrepreneurship center in Stanford School of Engineering and then basis the Business association of Stanford Entrepreneurial Students. I am so excited to have Alex Wang at ETL today. It's also his birthday secretly and so he chose to spend his birthday with you, which is the coolest thing ever. Alex is the founder and CEO of Scale AI. He founded Scale AI at the age of 19 while he was studying at MIT. His central insight was that machine learning powered data labeling could really help human beings make better use of AI. Over time time under Alex's leadership, Scale has grown to a $7 billion valuation and serves hundreds of customers across so many industries from finance to E commerce to US government agencies. Welcome Alex. Let's start off with what Scale AI does. I mean you've built this incredible company in three years. Tell us more about the breadth of work that you do.

First of all, thank you so much for having me, Emily. Really excited to chat today and I really enjoy chatting with you, so I'm excited. So yeah, Scale. So our mission is to accelerate the development of AI applications. You know, we believe that AI and machine learning is, you know, if not the most important technology of today, one of the most important technologies that's going to enable huge amount of goodness for, for the world and just enable the world to operate significantly more efficiently and effectively enable new customer experiences. And our vision in accomplishing that is building the most data centric infrastructure platform for AI and machine learning. And the real insight that we have or the thing that powers everything that we do is this thought that data is the new code and that the thing that will dictate the performance of these machine learning systems and these AI systems in the future is actually the data sets and the data that they're trained on much more so than the code that is written to power them. And so you know, if we were to boil it down to a two sentence, it's that better data results in better AI. And we've taken that sort of core idea and use that to build out an infrastructure platform to power a large number of a large swath of the sort of like AI ecosystem or the sort of like AI use cases out there. So we originally started with data labeling or data annotation which is the problem of converting of raw data feeds to useful label tag data that can be actually used to train large scale machine learning systems. We actually started in Autonomous vehicles. And then since then we've scaled across a variety of different industries like you mentioned, from E commerce to financial services to the government, to work with large tech platforms and sort of everything in between. And what we do with these customers is we help them not only with data annotation, data labeling, but also we help them with data management. We help them build out actually algorithms. And so with some customers we provide algorithms directly to them for stuff like document automation or E commerce, AI or in the government use cases. And so we've been able to expand pretty rapidly into a huge number of product areas, into a huge number of verticals. But again, centrally at the core it's all been powered by this concept that better data results in better AI. And the most valuable thing that we can do to ensure that we have great AI systems of the future is to build incredible systems to build great data sets. So that's what we do.

That's amazing. And the fact that you came across the seedling of that insight when you were 19, you weren't even 20. And I'm actually curious if we just go back now knowing what skills scale AI has grown to when you were 19, what gave you confidence in this concept that data is a new code? That was the basis, that was the soil for the growth of beautiful AI algorithms in the future. And what gave you the courage to jump in and do this?

Yeah, so it's sort of a few things. So when I was actually I was at, I was at mit, I was studying AI and machine learning and this was the year when Google released TensorFlow and, and DeepMind released AlphaGo. So sort of this like big seminal moment for AI and machine learning. And it really felt, you know, was this, you know I actually remember there's like this reporter who, who was walking around MIT campus and was like interviewing MIT students to see what they thought about, about AlphaGo. And so it was like, it was very clear moment where like felt like hey, AI is actually is going to happen, it's going to be big. And then I remember both in, in some, in some projects as well as like in some school projects. Well, some side projects I, and one of these side projects I remember very viscerally was like want to build a camera inside my, inside my, in my semi fridge that would tell me when my roommates were going to, were stealing my food. And I remember very viscerally it's like hey, you know there's all these great neural networks, they're really, really cool. But at the end of the day the Algorithm is only as good as the data that it's trained on. And so it was like, hey, this is a, this is going to be a critical almost pillar of, of whatever AI looks like in the future. And, and, and I looked around and I realized like, hey, this is a big problem that there aren't actually that many people trying to solve or there aren't that many people focused on solving this problem. And ultimately the thing that gave me conviction was frankly, I'd seen sort of the success stories in sort of the years prior of platforms like aws, which enabled everybody to build these large scale Internet systems or sort of like websites and large scale Internet platforms. Um, I've seen this discussive platforms like Stripe for, to enable payments and enable sort of like you to build businesses on the Internet. And so ultimately the realization was kind of, hey, you know, what AWS has done for the cloud or what Stripe has done for payments, you know, there's an opportunity for a company to do that for AI and unlock this huge amount of potential for the technology by solving one of these critical pillars. And the pattern recognition was that, hey, data centric AI was going to be just as important as some of these other sort of like foundational pillars. And so that's really what kind of got me excited at the time. And honestly speaking, I didn't have all the answers. I didn't know necessarily that this idea was definitely going to be as important as I think we believe it is now, or I didn't necessarily know that it was going to be as exciting as we think it is now. But the sort of fundamentals were there for it to be like, hey, this is certainly worth exploration.

I have a very important question for you. Did your roommate stop stealing your food?

Well, the punchline is actually I couldn't build it because just building that would have required so much data about different foods and different configurations and what it looked like to pull food away from the fridge if you. It was just like literally the amount of data would have been astronomical. That like, it took me all of two hours to just give up on that idea and realize there's no way.

Building on this question though there's more. Seriously, other than selling the food, I hope they didn't. They stopped your food. Regardless, you actually had some other entrepreneurial sort of experiences. You had some other things that you were building with classmates. And I'm curious if that sort of helped you feel out what the entrepreneurial journey was like.

Yeah, I think that there's like, you know, I often tell I Tell everyone this, which is that I think this sort of like this creation process of having an idea in your mind and then working with people to like, realize that idea into. Into the world. It's an incredibly empowering process. It doesn't always work. Obviously. It's not always the case that like, you know, build something and it's amazing, but I think it's like, it gives you so much confidence to like, go from literally an idea that you're dreaming up and you. You're sort of like working on an ice. You know, you're literally dreaming it up to go through this sort of like, process of making that a reality and identifying what are all the components we need to build, how do we architect it, how do we build that and actually build it into. Into a reality? Because it gives you so much. It gives you so much power as an individual. It's like, wow, I can. I have this ability to, like, will things that. From my. From my mind into. Into my. Into reality. And I think that. And I think that's. That's. That's, you know, that is like one of the most valuable things that, that you can. You can gift to anyone, right? And we actually, we have a value like this at scale. We call it ambition shapes reality. And maybe one day I'll sort of publish, like, how we think about this internally. But. But the core idea is that like, you know, there's this incredible quality where if you, you know, doing something that's ambitious versus not ambitious, you know, they might take literally the same amount of work. And so if you, if you, if you. If you are able to like, dream up big ideas and dream up big solutions to problems, and then you work to build them, you can have just like this incredible sort of like, feedback loop with the world and building great things and continuing to, to sort of like expand your horizons.

I love it so much. I think there's two things that I took away from what you just said. It's. I think a lot of times, especially as students, it's like, hey, you know what? I don't have the skills to build that yet, but sometimes it's really powerful to just get going and to realize that you can manifest, right? You can code, you can build an app, even if it's not perfect, you can manifest from an idea in your head into something. And just that prototype is very empowering. It shows that we have that agency to manifest and manifest and manifest. And then the second thing that you mentioned actually made me sort of think about like, my own journey in Food. It's like, wow, you know what, if I aim for like 100% better, I'm only going to get maybe 80% there, Lucky, maybe I get 100%, but if I aim for like 1000% better, I might get 500% right? So if we aim further with our ambition, we actually get further. If we set our finish line too close, it's too easy. We actually shortchange ourselves in many ways. So I hope you write that book, I really do hope you write that book because it's a powerful message.

It's so true. There's all these examples. I collect examples of this in real life. And one of the best ones is that like, you know, the four minute mile, for the longest time, humans thought that the four minute mile would have been impossible. And then somebody broke it and then all of a sudden all these people broke the four minute mile. And it was just this incredible moment of human achievement.

So same thing with high jumps, right? You know, like forever, people were like going over like front first and then some dude was like, no, I'm going to go over backwards. And then they had this step change and just the amount of height you could get by high jumping. So. Love it, love it. You know, I want to get to the crux of what your insight was and talk a little bit about data because oftentimes I think, you know, in the world everybody rushes to AI and then they forget. Actually the foundational element is the data, as you noticed. And you've done such an incredible job making that more accessible to everyone. So, you know, one of the, a couple of things are sort of top of mind for me in terms of challenges in the data space. So the first one is a question of data sparseness. So, you know, some industries that have been digitized over the last 10, 20 years, you know, lots of data flowing through, right? You know, financial industry, lots of sort of digitization there. But like, there's other industries like food and ag, like, hell if I know how many farmers out there are willing to spend time putting sensors in their fields to gather data. So, you know, what are your thoughts in sort of making sure that we bring along the industries that may not be as digitized, you know, how do we capture, help those industries capture more data or maybe find ways to use less data in order to build algorithms or other ways that you've thought of?

Yeah, totally. Yeah. And I think this is actually one of the fundamental, fundamental problems that we hope to solve. And I know like, even without scale is going to be solved over the course of the next, let's call it decade, because fundamentally you're exactly right like this. The, the way I think about this is that there's sort of like these two waves of technology that are sort of like crashing through the world, so to speak. The first wave is sort of the Internet and more and more computing and more and more sensors. And uh, that's great because if you have more sensors and you have more, the Internet is more distributed, you have better coordination, data gets saved more easily, you have automated workflows. Like the soft software is amazing for, for a bunch of reasons. And then like this interesting side effect is that as that's happening, you, all these pools of digital data get collected in, in all these interesting industries and all these interesting areas and verticals and then those pools of digital data are actually what creates the opportunity for machine learning systems or AI systems to then have huge amounts of impact. And sort of the second wave is, is AI and machine learning that's going to, that's going to build off the shoulders of the giants of like all this technology and sensors that have been deployed in the past. But then, but then you enter a problem which is that like, so if you think about the sort of quote unquote old use cases of AI, like large scale ad systems or search or ranking for social media, you have those, those use cases are very special because they have what are called natural labels. You know, just by us clicking on things on Google or just by us clicking on a video in our YouTube feed or clicking on something in our Facebook feed, we're providing labels about what are the things that, that interest us, what are the things that, that, that we're going to read or spend time on or look at. And the reality is in almost every other industry, in every other part of the economy, you don't have that. You don't have these natural labels that are just automatically feeding into these machine learning systems. But you have lots and lots of data. And so most industries and most companies, whether that be in the health care space or in financial services or agriculture, like you had mentioned agriculture and food, or in sort of climate change, there's huge amounts of data. These companies actually have lots and lots of data. But it isn't, what we like to say is it isn't AI ready, it's not annotated, it's not cleaned, it's not ready for the process of machine learning. And so that's sort of, in many ways it's like step one of what we've built at scale is how do we enable every business to go from huge pools of unstructured data to towards well labeled und biased great high quality data sets that are gonna allow them to build great machine learning systems of the future. And I think that like you know, realistically speaking there's, that's gonna take some time and that's gonna, there's lots of industries but, but that's like this, this sort of like, that's almost as like critical step or this critical bottleneck for AI to be deployed in significantly more areas. And the example I'll pose here, or the example we talk about a lot internally is if you think about imagenet, which was sort of the first really large scale like data set, computer vision data set that Fei Fei Li's lab at Stanford helped create. You know, that was the thing that then led to Alexnet and convolutional neural networks and all these deep learning approaches, us realizing that they actually could be really effective and they could absorb all this data. And so there's this really interesting thing about how data will lead to just having data alone is going to lead to really incredible machine learning and great techniques being used. And that's sort of one thing that we get really excited about. The other thing that I'll mention is there are plenty of use cases as well where maybe it's difficult to collect data, whether that be for privacy reasons or that's for lack of sensors or whatever reason. And, and these use cases do exist and sometimes they're incredibly important. And that's where sort of approaches around synthetic data or data augmentation become really important, which is how do you take what, what little data that you have and, and really supercharge that to be able to make it really, really valuable for building machine learning systems. And that's really something that we're, we're very passionate about about at scale as well is like how do we use synthetic data? Data augmentation to enable machine learning teams to combine the benefits of real world data with diverse and realistic synthetic data to ultimately achieve sort of really great machine learning algorithms. So it's kind of a mouthful. But, but data, data, data, lots of exciting stuff.

Yes, incredible. For some of the students who may not necessarily be CS majors here, I do want to talk about what quality data means and what data annotation means and it's so fascinating to me. Last week we had a speaker come in and talk about data in agriculture and how before artificial intelligence there must be human intelligence. So hi, before AI and she's really built her company at grow intelligence by hiring incredible experts who can interpret the data and really connect the dots appropriately. So with your work, maybe I could also point out a personal example that I'm fascinated by. Do a lot of work annotating food data and the chefs can tell the difference between a cilantro stem and a parsley stem. And I'm like, I have no idea. They look exactly the same to me. How do you work with your taskers and how do you train them to annotate huge amounts of data in partnership with your customers? Because they're not necessarily experts in transportation or experts in, you know, like, melanoma is on skin. Like, how do you then sort of help them train up so that they can do the work to ground truth?

Yeah, yeah. And so a few things, few things here. You know, one of the, one of the fundamental beliefs that, that we have at scale is that, you know, there's sort of like, there's sort of like a few resources that go into producing really high quality machine learning systems. If you think about like, what are the raw ingredients? One of those raw ingredients is compute. One of those raw ingredients is data. And then the last raw ingredient is sort of like human insight, so to speak, or like, and sort of the process of producing annotated data is combining raw data and human insight together and sort of, you know, mixing those ingredients together. And you know, all AI systems of the future are going to be heavily reliant on human insight, even if, you know, we have incredible advancements in the technology, etc. Because at the end of the day, you know, we're at minimum, we're going to need oversight in these AI systems to make sure that they're, they're built, they're producing the results that we would expect and producing results that we think are, are make sense and whatnot. And so I think, we think this like, problem, this sort of like one of the core problems of machine learning is like, how do you make sure that for every problem that, that you might have, that you might want to solve using machine learning, you might want to have a machine learning system do. They're able to effectively get human insight to sort of like power that power that use case. And the way we think about it is like, is twofold. I think first is for folks who are trained up, you want those individuals to have as much leverage as possible. So you want, and this is somewhat circular, but it becomes very important. You want machine learning systems to do as much of the work as possible and then really have humans spend all Their effort on almost quote unquote the edges or on the really like high judgment work that's required that enables the utmost quality while making them the most efficient in their roles. So that's the first piece and the second piece is sort of this, this general education problem around how do you, how do you enable people to be experts and how do you like train them most efficiently? And one of the almost funny side effects of, of solving this problem at scale is that I think we, we have one of the, probably the more interesting ed tech systems out there in which we have systems internally which track all the different kinds of, you know, edge cases might be one way to talk about, but all the different kinds of nuances to some of these, these data problems, right? Like how, like this, this cilantro versus parsley example could be one of them. And the different ways in which you can tell or detecting a melanoma versus other kinds of, other kinds of growths on, on, on skin. So you know, there's all these different nuances and we track all those, all these different nuances within our internal systems. And we are constantly trying to understand for each of the annotators what is which, which nuances or which edge cases are they performing well at and they understand super well and which ones might be tripping them up and then proactively serving them materials and content and examples that help them elucidate these, these cases that they might not understand super well. And so it's, it's this, it's I really think about as an ed tech system which enables, enables people to really quickly grok and understand all the nuances of the data. And humans are incredible at pattern recognition, right? If you think about what it takes to go through medical school and you talk to folks who have gone through medical school, it's an incredibly long process of just sort of continued pattern recognition. And that's sort of that same process where we try to distill for the annotators in our system to enable them to do great work.

That's incredible. So first and foremost it's interesting that you've actually, it sounds like you build AI to then help your annotators get better faster, which is recursive, right? You're actually using your own product on yourself to make yourself better over time. And a similar vein, I think for a lot of students, AI can help make AI better, right? So oftentimes, you know, we talk about, you sort of talk about it. We want the humans to pay attention to the thing that actually matters on an Image or you know, in a sentence. And AI actually can be used to segment or pull out the part that requires attention. And so again, there's all these recursive loops in here that I find totally fascinating with, with your work and let's maybe broaden out a little bit. You know, we had just kind of touched upon the melanoma example and something that I find fascinating with sort of a lot of AI systems that are looking at skin, for example, are focused on what's available already. Right. So algorithms currently, as I understand it, are much more accurate for fair skinned individuals because there's just more data. So you know, when you think about like helping your customers that you call your partners sort of build sort of fair, balanced sort of AI systems, how do you have that conversation with them? I know it matters less with roads and maybe like license plates where it's pretty unambiguous when you, when you look at numbers. Right. But like how do you have that conversation with a partner when they're solving a problem and to aim for being responsible?

Yeah, definitely. Well, I think first off to your point, I think responsible AI is, you know, is incredibly forward to have responsible AI. It's incredibly important to build high quality and representative data. You know, I actually think it's, it's critical because at its core this thing I talked about earlier is that, you know, the data is almost like the food that you feed to these machine learning systems and ultimately you are what you eat. And so that the sort of, the data is sort of the, one of the fundamental areas where these biases or these kinds of really critical real world issues can sort of stem from. And really the way we think about is like how do we build technology, how do we build tools, how do we build things to enable either our customers or ourselves to ensure minimal bias and minimal harmful results with the data sets that power these machine learning systems and, or how do we build systems that are going to flag that or identify when those biases may exist as proactively as possible so that then we know that we have to go fix that problem long before it ends up in the hands of a consumer or some sort of critical decision making progress. So for example, we recently worked with a bunch of medical researchers on automated medical imaging analysis, this sort of exact problem. We worked at the MIT Media Lab and analyzed this exact problem that you mentioned of tons and tons of clinical images and found there were a lot more light skinned images than dark skin images. And what we did is we actually basically helped them replenish or sort of debias the data set and, and, and, and add more data to the unrepresented classes, either with real world data or using data augmentation or synthetic data. And we're able to significantly de bias the output or the outcomes from the machine learning algorithm. And so I think, you know, it's, it's not an easy problem in any way. You know, this is actually a sort of like, you know, a, this is a one of the quote unquote hard problems of AI in the sense that there's no easy solution. It's not like, you know, I don't expect that next year we're going to get some, you know, crazy machine learning architecture that all of a sudden solves the problem of data bias or machine learning bias. It really is all about the nitty gritty details about what does your dataset look like, how do you know ahead of time when there's bias in the data set? And then how do you fix that or how do you proactively resolve that and how do you keep going through that process?

You know, in some ways that was a trick question because in order to unbiased, you have to be aware of the bias. And so that is not always easy. So I know on the back end, you know, you can unbiased. Once you're aware of the bias, do you work with your partners to gather more data? In the case where your sort of initial preliminary analysis is that there may be bias, you go back to them and say, hey, you might want to collect more in order for it to reshape this?

Yeah, we do. And in fact, a lot of times, and going back to this, like a lot of times our customers, they actually are sitting on incredibly large troves of data, but they don't have any of the tooling or infrastructure to help them sort of sift through the noise and identify quote, unquote needles in the haystack or identify the particular pieces of data that are going to drive forward their model performance or drive forward bias the most effectively. And so we built tooling, our Nucleus product, which actually enables this exact process of like, hey, I know there's a problem with, let's say, not enough of a particular kind of skin growth in my data set. And then how can I then go through all my unlabeled data, all my entire large troves of data to just get more samples of this kind of skin care skin growth, so that my algorithms are performing significantly better in the future? And that's really the sort of like much More so than even, you know, getting the data. It's about, it's about put sort of instilling this sort of philosophy around continual improvement of algorithms through continual improvement of data. And I think that, that if we can get there, then I think we're gonna be able to cope with all these challenges, with bias and data issues and all that stuff, model issues, much more effectively.

That's very cool. Very cool. So maybe I could zoom forward. We were starting from when you were 19 when you came up with Scale AI. If we were to sort of look out to 2032. I know you've worked in many different industries, healthcare, transportation, real estate and more. What are you excited about? Like, where do you think we're going to be in 2032? And, you know, with that, the secondary question is then, you know, given that we have a lot of budding entrepreneurs in our class and beyond who might be watching on YouTube live right now, whether it's going deeper into picks and shovels, you know, as you've done with scale AI, or other places within the ML gold rush that are like, really, really interesting, like, looking out to 2032, where do you think we're going to end up and what are the opportunities?

Yeah, no, I think it's. I mean, I truly think, you know, we're kind of in the golden age of AI, so to speak, where the sort of proliferation of use cases is going to be absolutely massive. Like, even at, even at scale. We started in autonomous vehicles and the first few years of the company, you know, it actually, it felt like machine learning felt kind of, I don't know if the word is lonely, but it was like autonomous vehicles were the big use case. Everything else almost felt like, you know, a sort of side project or something much smaller. And then fast forward to now, we see all sorts of like, really exciting use cases in every single industry. So with financial services customers like Brex or PayPal or, or Square or whatnot, we see interesting use cases around trying to understand, build systems that move money more effectively or identify or understand transaction flows a lot better, or identify fraud much better. Or we see use cases with Flexport, which is sort of a global trade platform, and enable just like an incredible amount of efficiency in the process of global trade, which, which is very important. It's really important that we're able to get goods delivered from everywhere in the world. And it's an incredibly manual process state. And by using machine learning, you can automate a huge number of those workflows and enable the overall economy to Just get a lot more efficient or whether it's with, whether it's with a large scale automotive company and a car company and building not only full stack autonomous vehicles but also driver assistance systems or systems that make drivers more safe, etc. So I think that like we're in this phase where there's massive proliferation of, of, of machine learning systems. And I think that like the, the way, one way I would think about it is you know, that the, the sort of like software eats the world mindset is that you take like, think about any industry and any, or any sort of like any problem in the world today and just imagine, okay, if you had software, how could, how could you transform that? And I think that we've just seen this has been this like very long term sort of slow transformation because it turns out humans are sort of like infinitely creative. And you'll take any system and we'll be able to identify oh you can use software in this way or oh, you can use software in this way. And then you even sometimes even replace existing software with new software. Like, you know, there's like lot like old enterprise systems are replaced by like new style, more consumer kind of, kind of Internet platforms. And so, and so I think it's going to be this continual process where we're going to look at something, we're going to look at a problem, let's say like in insurance, the process by which claims get processed. We're looking at a problem and we're going to think, okay, if you act like if you use machine learning in the right way and not just AI in some magical sense, but actual like you know, the core, core fundamentals of machine learning, then you can design this process to be 10x more efficient. And we're just going to keep identifying all those problems. And I think you go Fast forward to 2032, it's going to be everywhere. It's going to end and the opportunity won't have stopped. We're still going to have plenty of opportunity to apply AI to these systems. I think maybe more, to name a few specific examples because I think these are maybe some of the ones that are quote unquote cooler or more exciting right now. I think that there's, there's a few that I think are really important. And for those of you with who are sort of like thinking about what ideas are exciting, I think these are, these are maybe some, some areas to think about. I think first one is science. Science is, you know, there's actually these papers about how scientific progress has kind of been slowing actually a little bit over the past few decades. And, and one part of that is that like you think about science, let's say a century ago or two centuries ago, you could do so many experiments and your ability to validate your ideas was, was, was really, really exciting. And now we're at a point where like a lot of the cheap experiments, so to speak, have been explored and now we have like ridiculously expensive experiments. You know, like particle accelerators are extremely, extremely expensive or large scale clinical trials are very, very expensive, et cetera. And one of the really exciting use cases of AI is using AI to simulate effectively, basically simulate experiments significantly more effectively than you could in the past or they could using classical methods. And there's already a lot of examples of this in whether that's something like an alphafold out of DeepMind or, or using AI applied to sort of like fusion experiments or fusion simulations. But I think it's gonna be a huge boon in physics, chemistry, biology, pharmaceuticals. And it's going to transform a lot of the, sort of like it's going to be this base technology that empowers a lot of future innovation. So I think that one's really critical. Metaverse is a use case is something that a lot of people are talking about these days. I think it means a lot of different things to different people. But I think if you think about AR or augmented reality, which is probably one of the form factors that probably feels most intuitive to a lot of us, which is that hey, we're just going to have this digital overlay over our natural lives. If you think about that problem, it is an incredibly complex machine learning problem, an incredibly complex AI problem because fundamentally you need to understand the world and how these different objects relate to one another and how I as a person can relate to those objects. And, and if people are walking past each other and they make a look, you need to be able to understand that kind of stuff. And so you need to have this very fine grained understanding of, of what is going on in the world around you. And, and that is a very, very challenging AI problem. But I think that it's one that you know, is, is going to enable these sort of like very sci fi like consumer experiences that I think of the future that I think we're all really fundamentally really excited about. And the last one that I'll kind of mention just because you know, I think, I think this one is really important and it's, it's somewhat controversial, but I think it's an important one to talk about is, I think AI is applied to the government's problems, in particular applied to sort of national security, defense intelligence, et cetera. And I think that we're in a very interesting period in the world where the, the, the, the sort of like warfare is shifting from, from sort of a previous paradigm to a significantly more digital paradigm. And now, you know, a significant number of, of these sort of like skirmishes of the future are going to happen entirely digitally in cyberspace or via AI systems or via purely digital systems. And I think it's really, really important that if, you know, if you believe in kind of democracy and you believe in the values that the United States represents, that the United States and other democratic countries are able to utilize best in class technologies to not be vulnerable as a sort of long term platform shift is occurring. And so I think it is really important that we have some of the best and most brilliant technical minds thinking about how do we build the best in class systems for the future of the United States to enable the United States to be as effective from a, from a defense and sort of intelligence perspective as it has been for the past, you know, call it 50 years, which has really enabled the sort of like modern era of peace. So I think those are some of the areas that excite me the most, I think are most important.

Incredible. So to sum it up, science, metaverse, AR VR and then government. It's fascinating to me, if I were to tackle that question, I'm almost like, what are the chores that human beings don't like doing? Like washing dishes or like dealing my taxes. Right. Like one day, all the things that are taking up our time and could be automated. I see 2030 as a point in time where all those things are automated and we are free to actually do what only uniquely humans can do, which is to spend our time being creators and being creative. And I hope that that's the future that I look for and I think I would support the comments that you made here. Let's talk about robots for a second. At one point in the early days of some of the everyday robots work at Google, I would have a robot come to my desk and try very hard to clean up my desk, and it would fail. Generally we try to pick up my bowl of cereal and accidentally pour milk all over my desk. Or like once in a while it would like steal my bottle of wine. You know, like all stuff like that. Putting that aside, I'm actually a little bit curious about, you know, with AI and data and you know, sort of tied up the conversation that we had around ar. You know, in the future there's going to be, you know, artificial humans. Right? And, you know, I heard an interesting podcast recently with Ezra Klein interviewing Ted Liang, who's a science fiction writer. And you know, he said that, you know, before machines become sentient, we will probably end up causing them to suffer a whole lot. And you know, this kind of begs the question, you know, AI, how do we, how do we give feelings to machines? And are machines going to have feelings? And is that a bridge too far? And you know, I know we're sort of steering away from sort of data, but the whole entire purpose of the work that you're doing is to really create this foundation for data to then bring AI, you know, into the future to make human life better. Curious how you're, how you feel about robots and machines and that space.

Space. Yeah. No, I think it's a super interesting philosophical question, which is that like, you know, let's take a, let's take a, you know, right now in a Stanford AI class, most of the students will train some sort of simple neural network to recognize objects and imagery. Is that neural network a, Is that like, does that neural network have feelings like when you will delete it from your, your hard drive because you're running out of space? Is that, is that, is that, is that, does it, does that cause suffering to that neural network? And I think that, I think these are super, you know, these are, it's like very tough questions to answer, like where, where are the boundaries? What are the, for each of these edge cases? You know, what does that kind of look like? I actually think that, you know, I think the sci fi fantasy states in the future where you have AI systems that actually fully resemble humans in terms of their ability to have judgment and you know, and, and communicate with us? I think it's actually quite far. And you know, my, in my evidence point is like, we're still not at a point where an AI system can fully accurately process documents and ocr. Like still like, does not work fully effectively and the judgment that goes, the judgment that will be require from AI systems actually like resembles people is so far off that I think that, I think we're going to be in this sort of like limbo state, so to speak, where, where we have AI systems that are very effective and very useful and can do things that like, maybe we didn't think they could do like three years ago. But you're not going to get to the point where you have AI systems that are, that are, that you Would you would sort of like, you know, where there's like these really deep philosophical questions that are, that are tough. One thing that I do think though is, is going to happen probably sooner rather than later. And this is maybe the sort of like near term problem is that we're going to more and more have AI systems that affect how humans think. Right? And, and this is sort of like it's already happened to some degree where social media systems or ad systems or, or even search, you know, these are, these are machine learning systems. They pick content for you and that, and the way they pick content for you will change how you think. And this is only going to keep going in, in that direction. Like I think in the future we're going like not so distant future, we're going to have children, are going to have sort of like AI friends and they're going to have these chatbots that they can talk with and practice, you know, speech with and sort of like, and learn from and talk about their days and like basically like true friends. And they're going to affect how, how humans think, just like how you know, phones have sort of like changed the human's attention span probably permanently. And so I think that those, there are sort of these like interesting social issues around what does it mean for AI to have such a, such a real tangible impact on how we might behave in time? That I think are, I mean those like that's a really important sociological question and what are the implications of that? How do we get ahead of it? And I think ultimately, at least from our neck of the woods and our view, I think a lot of that comes down to again, how do we ensure accuracy, quality and efficacy of the data that are feeding these ML systems so we never end up in like a really, a really bad scenario where the data distribution or the data used to feed in one of these algorithms was so off base that it caused all these unintended consequences of the sort of the people that were using them. And in some sense I think that like, you know, if you think about sort of like political polarization that's happening on some level that is a data distribution problem. It's like, hey, you know what's happening is that these ML content systems are favoring more and more polarized content and the data distribution has shifted so significantly and that's causing all these like weird social, social dilemmas that, that are, that are, that are really tough to think about and tough to deal with. And so anyway, long story short, I think it does all come back to data at least, like, I think data is a, is a necessary but not sufficient part of the problem. And I think that AI is going to change how we think pretty soon and we're going to have to think about that.

That was fantastic. Thank you for being so. Just genuine in responding to a pretty provocative question that I was asking you. I know that the students are hankering to ask you questions, so let's dive into that for the final remainder of this time. For those of you who have not already upvoted, we're going to start with the questions that have the most upvotes. So please get in there and upvote. First question here. What are some key challenges you have had to overcome while running your company? Bringing back to life.

Yeah, yeah, no, so it's a great question and it's almost, you know, answering is almost impossible because it's almost like what, what challenges have I not had to overcome in building a company? I think that one of the, one of the things about, you know, I think it's probably like doing, doing anything challenging and sort of this, this concept of like, you know, whether it's building a new product or building up, building a company or, or building a team, you know, there's just, you know, it's like whatever, like you'll deal with so many challenging scenarios in time and there's sort of like this infinite chaos to the world that will cause you to sort of be. Be confronted with the most strange circumstances that you, like, could not foresee, and you're just going to have to figure them out. But I think that, like, you know, if I had to, if I had to distill it down, I think probably the, the parts about building scale that have, that are really maybe the most challenging but also most rewarding have been about building the team, getting great people to sort of like join the mission and really, and really see the potential of what we'd be, what we're able to accomplish. Whether that be, you know, whether it be people who I work with or the incredible team at scale, whether that be our amazing investors, whether it be our amazing customers. But I think this sort of like this fundamental challenge of getting people to buy into what you believe the future to look like, I think is one of the hard problems of the world. And I think I felt very lucky that so many people have chosen to take a chance on me.

Well, you have an incredible vision and a lot of clarity around that vision, so I can understand why a lot of people have joined you in whatever way they can. I think that's also underestimated. I think a lot of our engineering students here, like, oh, you know, I'm going to spend a lot of time on the tech, but it turns out that spending time on people is just as important, if not even more important. All right, next question. At what point during the development of your company did you realize that, number one, it could become a serious competitor in the AI space and number two, that you were ready to pursue building it full time?

Yeah, no, it's a really, it's a really good question. I think that, I think that early on it always is helpful to be wildly optimistic, I should say, or at least maybe the proper terminologies to dare to dream big, which is that even early on I think this sort of like this thread of thought which was, hey, if we actually do solve this sort of like data problem for, for the machine learning community, that is big, and I don't know exactly what it's going to look like to build from that point, but we're going to be able to do a lot of interesting stuff if we get there. And so that thread of idea did emerge and it didn't become. It honestly wasn't relevant for us for the first many years of the company because we're just focused on data labeling, data annotation, we're just focused on like solving that. But then you sort of, you know, we solved that problem. We really focused on, we got scale and then we picked our heads up and we're like, wow, we're actually at this sort of like the promised land that we thought at the very beginning that like, we can, we actually have a shot at like building a lot more, doing a lot more and more industries to sort of accelerate AI and machine learning development. And so, you know, I would say that it was a. Maybe early on it was a pipe dream. And then it was sort of like, you know, many years later before it actually felt like, hey, this is like a real, a real opportunity for us. And the second part of the question, which was, at what point did I decide I was ready to pursue it, building it full time, I actually like, I wish I could say, oh, I was just like, so convicted. And so I just. And so, and so I just like went for it. But actually it was, I really started working on it in earnest when school ended. I remember even like, I put, I put working on all of it on hold, what to get through like my finals and my final projects and all that stuff. And then the summer started and I was like, all right, I have Nothing else to do. And then. And then I was. We're very fortunate that we got, over the course of the summer months, enough momentum where it was like, hey, there's kind of no option but to just, you know, keep going at this thing. And so I think it's. I think it's sort of like this barrier, this mental barrier feels very large oftentimes around like, hey, I have a life right now. At one point. Am I going to be ready to, like, sacrifice my life today and then. And then go do. And then go approach this other life? I think the. And I think a lot of times I'll still seem very irrational. And my. Maybe my advice is to like, not. Not actually be quote, unquote, trapped in that false dichotomy, but actually just put yourself in situations where you have sort of freedom and flexibility to explore and then, like, see how you gain momentum in that scenario.

I appreciate that because sometimes students feel like they have to have something lined up for the summer, but it was actually a gift for you not to have had something lined up and to have that space to kind of really dive deep into something that you really cared about. Similarly, let's take the next question here. Knowing what you may know now, after running your complete for three years, you could go back to the first day you started your company. Would you do anything differently?

Yeah, well, there's always a butterfly effect where, like, maybe if I did something slightly differently than like, that, you know, all these other things would have gone. Gone totally differently. And so on the whole, I wouldn't trade sort of. I wouldn't, I wouldn't. I wouldn't. I wouldn't screw it up too much. That being said, I do actually think that, like, you know, the biggest thing that, That I would say, like, if I were to go back in order to tell myself one thing, it would be that, like, you know, it's. The people matter so much. And like, you know, however much, you know, I was, I was really focusing on people early on, and a lot of that was. Was guided because of mentorship that I had gotten or advice that I had gotten. And also just the sort of like this, like, selfish desired was like, hey, I want to work with people who, like, really inspire me and really get me excited. And that's, That's. That seems like the best way to live, especially I'm going to be working on this all day and night. And so that was like, you know, I ended up, you know, getting there a little bit, but I would just. It's like now I know in my mind state that it's like however much I thought that people mattered at the start, they matter like 10. It's like obviously they matter 10x more now. And I wish I could sort of like transport that thought back to my, back to my prior self. So that, that's kind of tough. I also think maybe the other piece is like along the way there's, there's a lot of moments of like great self doubt and, and that's just, it's almost like part of the journey because almost invariably you're just gonna end up in a situation where it's like, wow, crap. I've almost like no idea what to do. And um, and I think probably the big thing I would tell myself is like, it is like totally okay to be in that spot. It is totally okay for that to happen to you. Um, it is not okay to be paralyzed when that happens. And, and even if you, you, you literally take the worst possible action coming out of that scenario, it will be better to have taken that action than to have taken no action. And so, and so maybe that's the.

Other thing I would tell myself ultimately there are no, I would say one way doors. Right? I think by taking an action we can always course correct along the way. But if we don't take any action, we don't have any new information. And so it's super important to actually make a decision, act even with imperfect information. All right, next question. With recent scrutiny regarding data privacy, how do you think AI companies who are of course reliant on good data, should go about data collection?

Huh? Yeah, no, I think this is a super, is a super important question because I think that the question is like, and this is spurred in. You know, there's been a bunch of examples in the past that have spurred this, but some of them, for example, are like, you know, the realization that Siri or Alexa are learning from recordings that you. And they had no idea that they were necessarily learning from those recordings. And so I think it's like a super important, almost like reorientation that the entire technology or ML community needs to take, which is like going from having this thought is like, however I can get the data, I will use that data and then train machine learning systems on it to. I need to get this data in a very responsible way. I need to do it in a way that is, that is like where everybody who's providing the data understands that their data is being provided for these AI systems, where I'm collecting it in a debiased way. Where I'm collecting sufficient amounts of it, et cetera. And I think that, you know, the answer is that that's going to make, that's going to make it harder to build the same kinds of AI systems potentially. But it's going, it's the way that technology needs to develop. Like, it's not, it's not a tenable state for machine learning systems to be developed sort of by, by taking advantage, maybe like, you know, a checkbox that nobody reads or, you know, stuff like that. And so, so I think it's a very important shift. It's a big shift. It's going to happen and we're helping a lot of our customers think through that and, and make them robust to ensuring that they actually get data for these ML systems that is, that is sort of responsibly collected.

You know, it actually goes back to something you've said many times, even publicly. Like your customers, you partner with them, right? You're not just providing them with a solution, you're really, truly partnering with them and being in dialogue with them. Because these, these challenges like privacy are very nuanced and tricky and you can't, it can't be the same thing every time, right? So you have to sort of lean in and have the conversation. Conversation as a partner as opposed to selling a thing that's finished. And so I'm really glad that that question was asked because it is something that's top of mind for so many people. Let's tackle one more question. One more question. Try to fit it in there. What have been your most significant failures? Oh, goodness. And what did you learn from them? Or like, let's start with just one big failure that you thought was a failure. And how did you learn from it?

Yeah, you know, I think, man, there's so many. But I think I'll talk about one that is very much top of mind. We had this one, one of our first customers and really our first very large customer. And I won't name them, but they're sort of like they're a prominent company and they really, in the very early days of scale, they took a huge chance by working with us because like, you know, obviously our product was really janky. It only kind of worked. It took a huge bet on us to that we would be able to solve this problem for them. And, and actually for a while we had a pretty successful relationship with this customer and that's what allowed them to grow to be very large and allowed them to scale up with them and for us to sort of like, you know, to build this great relationship with them. And then one of the, I think it was like, I don't know if it was. I'd call it a failure or a mistake or whatnot, is that we, we ended up losing focus on that customer. And it was sort of like, okay, let's go find other big customers. Let's go do all this other stuff. And in doing that or in that process, we ended up, we ended up really like, you know, in the ignorance, in ignoring that customer. At a certain point, we just, we weren't actually solving their problems and we weren't actually sort of like being the, being the scrappy team that they could bet on, that they could actually like, bet on to build incredible product for them. And they ended up leaving. And I remember this was like this incredibly. It was very challenging. The moment. This is one of the examples of like, you know, when I talked about, hey, you're going to have moments of great self doubt. And I remember asking myself, like, hey, is this. Do we even have a business? Like, does it even make sense for customers to work with us? Like, asking myself all these questions and, and it was, it was a really tough moment because it was also one of these things where it's like, you know, there's almost nothing that we could have done to convince them to have continued working with us. So it was like, this was these one situations like what's done is done and, and the big takeaway or the big thing that we learned from it is like, it's just so, so important to be customer centric and it's so important to, to always be focused on your customers and make sure that you're, you're always understanding what, what you need to do and how you make them just ecstatic about your product and how you, how you sort of like exceed all their expectations. And that moment really, like, however embedded this concept was in our culture beforehand, it just, like, it just made it even more visceral and made it more obvious. And so I think that was an incredible learning moment. The only thing is, like, I'm very grateful that it came so early on in our journey.

That's a wonderful way to end. And you know, I appreciate you talking about these challenging moments as moments that really sort of forge our conviction. And that was such a great lesson that you had there. You know, I am so grateful that you were open to joining us on your birthday, Alex. It's just incredible and fascinating that you're doing this work. And I will be following scale AI for a very long time. I want to also thank the audience here for tuning into this session of the Entrepreneurial Thought Leaders series. Next week I will be talking to Josh Geigel who is the co founder of Virgin Hyperloop. So completely different industry and transportation. And if you are curious about future events, we have a website that's Ecorner Stanford Edu. You can learn more about further speakers. And then we also have a YouTube channel with videos, podcasts and articles and other things. So please take a look. And as always, thank you Alex. Again, happy Birthday and thank you to all of you for tuning in to etl.

Thank you so much Emily. Really, really glad to be here. And thanks for everyone for asking great questions.