Stanford Professor Andrew Maas explains why the future of AI will rely entirely on humans

Thank you all so much for being here today. Really excited to be joined by, of course, our lovely CEO Ali and our wonderful new VP of AI, Andrew. So to get right into it, we're excited to have this conversation and have all of you, all of you be joining us today. And so Andrew, would love to start with a little bit more on your background as you recently joined us. So before we get into the more technical discussion, would love if you could share a little bit more about your background, what you were working on previously, and the journey that ultimately brought you to us here at MicroOne.

Sure. Yeah, I've worked on deep learning for, I guess, quite a while now. So I started a PhD doing deep learning back in 2009, working with Andrew Ng at Stanford. And those were days when we hadn't yet convinced the broader machine learning, computer vision, speech recognition, NLP communities that deep learning methods were, you know, even worth taking seriously in some cases because none of the state-of-the-art results were there. And so I was very lucky in that the timing of my PhD kind of coincides with uncovering some of these things we now kind of take for granted. And have really honed over the years, which is that deep learning models are capable of absorbing larger and larger datasets. And they can do that in a way that enables them to really learn like pretty complicated stuff. And after my PhD, I actually started a company that was doing complicated data creation for healthcare annotations. This was before LLMs. But with the idea that a lot of what limits our ability to apply machine learning systems out in the real world is the real world actually has a lot of sort of like contextual, very specific domain-specific knowledge that matters or domain-specific things we want to predict. It's not just, you know, give me an image classifier for cats and dogs and I'm happy, right? And so the trend has been, you know, seeing models get more powerful and then seeing some of these data creation, data quality tasks getting more complicated. So that startup, we actually were acquired by a large clinical research organization. So we sped up by a factor of maybe 2 to 5x a lot of human experts who have healthcare and biomedical expertise who were scanning through drug safety reports, for example. And my career, I then got back into sort of large-scale deep learning. I worked at Apple building the data labeling sort of engine for a lot of their self-driving or autonomy-related research a few years back. And as things like ChatGPT and LLMs and the potential for enterprise agents became so exciting, I actually left Apple and started a company called Pointable. We enabled enterprises to develop some very specialized RAG chatbots, so pairing retrieval systems with LLMs in order to have product information, public chatbots that are very, say, faithful to a company's very particular information. That company didn't last too long. We existed for about 18 months. We were actually acquired by Apple to help with some of the Siri and speech technology efforts. And so that's what I had been doing most recently. But Ali and I met in the context of teaching a Stanford course, actually. So I had been keeping in touch with MicroOne as Ali sort of built from this really interesting sort of Zara recruiter into now a whole wide range of enabling experts and domain expertise to inject into some of these very prominent AI models that are out there. Because it's clear to me, having worked on all these different things, that there's a whole kind of new generation of data challenging and annotation challenges that are coming up. And because our models and LLM reasoning systems are now so much more capable, we have a new class of problems around how do I generate high-quality data and what does it even mean to be high-quality data and feedback? And so that's my kind of meandering path, you know, career-wise. But I've stayed really focused in a bunch of different ways, both academically and in industry, mostly on the startup side, on these issues of, you know, deep learning systems really work when we have amazing data and amazing enabling platforms. And getting that to work in enterprise often ends up being limited by getting domain experts to sort of work well with the process of creating machine learning models. And MicroOne, in a bunch of ways that are fascinating to me personally, is engaged in these types of challenges for the new generation of models. We can now get a lot more sort of value out of, out of human expertise. And so That's what sort of led me to MicroOne all along. Yeah.

Great. Thank you so much for that. And to flip the table, Ali, from your perspective as our founder, what stood out to you when you first met Andrew and started talking to him and ultimately talking with him about joining our team?

Yeah, so I, I actually messaged Andrew on ed when I was at Stanford. I, I made a private ed post saying essentially like, hey, you should consider joining us. And, you know, I think at first Andrew didn't really engage too much with this, but of course, over a long period after that, we were very grateful that Andrew has joined us. I think what got me really excited at first was the intersection of expertise and interests and experiences that Andrew has is like kind of the perfect match for MicroOne. Of course, the first part is teaching conversational systems and audio AI systems at Stanford. A large part of what we do is build Zara, which allows us to recruit experts and assign them to data pipelines and jobs. And so a huge part of our product is building these conversational systems, which Andrew is a legend in. And then I think there's two other aspects, which is Andrew has been in the game of human data pretty much longer than most people helping build, you know, as you mentioned, the data department at Apple, which back then was a lot more kind of generalist type labeling. And Andrew has seen the whole kind of trajectory of and transition into this expert level data. and I think that, that experience is incredibly valuable for us. And I think lastly, uh, you know, this is very relevant to you, Izzy, which is, um, we see evals really, uh, we, we need specific evals for enterprise use cases that are very contextual to kind of niche workflows within enterprises. And of course, Andrew has built a, a company around this and has a ton of experience in this. So I think the intersection of those 3 things was, um, basically made Andrew the, the perfect person to lead our centralized AI department that does all those things. So yeah, it's really, it's an honor to have Andrew join us.

Absolutely. Completely agree. It's a match made in heaven if you ask me. But so to get into a little bit more of the topic of our conversation today, of course, our event title is AI will be as human as we make it. And it's You know, the statement isn't necessarily a reassurance. It's really a responsibility. So I want to talk a little bit about what that actually means in practice, starting with you, Ali. You've said the goal is to have AI systems that are similar to humans but also aligned with human values. And the only way to achieve both is if they fundamentally continue to learn from humans. So my question to you is, what's the mechanism and what does that learning actually look like in practice?

Yeah, it's a great question. I think what it looks like in practice is essentially everything we do at MicroOne, which is distilling structured human judgment that really resembles the real world as much as it can into models. In ways that are obviously very aligned with actual human desires. But I think just to take a quick step back, I think there's 3 main pillars to what it means to make AI as human as possible. First is, you know, have experts distill knowledge in a way that is aligned with the long-term values, which are changing and drifting of society. I think that the second is treat the job, which is AI training, which is distilling this knowledge into models, as one of the most important job sectors of our lifetime. And, you know, this job sector is quite small relatively to others, but I believe it will be one of the largest job sectors in the world in the long, you know, long term. And so treating this job sector as a really important one that pretty much every domain will work in is sort of the second pillar that I think about. And then the last one is building models and agents in a way where humans will always be in control and really kind of building interfaces where the ultimate decisions will be made by humans and Of course, the, the one, the example that, that we like to use and, and we've designed this in a very specific way that kind of follows this principle is, is Zara, our, our AI recruiter, where Zara does a lot of things in a very autonomous fashion. It, you know, sources candidates on its own. There's autonomous messages that it sends. There's a referral program that results in lots of inbound. Of course, it does autonomous interviews, gives reports to recruiters on how candidates do. But ultimately recruiters, the human recruiters are the ones that do the kind of final layer sanity checks to then assign candidates to any given job. And I think, you know, this notion of end-to-end autonomous agents can exist with humans still being in complete control at the final layer. And I don't think it's contradictory to the impact it will have. I think there's this, you know, the counter to this is that, You really limit the impact if you always have kind of a final layer, uh, human review in some way. But I actually don't think that that's true in this case. You know, our recruiters are hiring hundreds of people every single day. Like one recruiter hires probably roughly 100 people a day, something along those lines. And so the impact is probably about 100x already, but still there's that final layer, uh, human review. So I think building agents with this in mind, um, is, is one of the most important pillars of making sure that AI systems are aligned with humans, but also very much controlled by humans.

Definitely. And Andrew, what does, you know, more human even mean as a design goal? Is it— do you see it as behavioral? Is it about uncertainty quantification? Is it about memory and continuity? Would love your take.

Yeah, I, when I think of more human, I think of AI systems that would do a task in the way that we would trust a human to do the task, which is a little weird to say because there are, of course, AI systems that we might trust a lot more, right? Like, I actually trust my lane keeping system on my car a lot more to just make those fine little motor adjustments than I trust myself at times. But in a lot of agentic workflows and with LLMs, we experience this information back from them, or we're sort of entrusting them to make multiple hops of reasoning. And there we get into this very non-human things where it's a— we might say like, well, it didn't consider, you know, the social impact of, you know, spamming 1,000 people as a marketing strategy. And so it proposed that, right? And so that's where we say like, it's not human enough, right? Or Of course, a lot of things with sort of safety and guardrails. This is not responding to me in the way that, you know, a thoughtful human conversation partner would. And so what's exciting to me about all of this is we have the technology in transformers and LLMs and these inference reasoning chains now to be able to match extraordinarily complex human behaviors, right? Like, you know, sometimes when you're using different types of like LLM chat models, it really feels like there's kind of a person thinking about what they're saying back to you. Right? And so we can capture a lot of really expressive behavior. And so to me, what it means to be more human is to figure out as an industry, how do we build this kind of, you know, machinery so that we can take the good parts of human expertise and get feedback and human evaluations of how these systems are thinking in order to match what we want them to do. Right. And I think this is a positive story. This isn't that we're going to go, you know, try to impose a lot of, like, safety guardrails or, like, tell a system what not to do. Actually, the bigger problem by far is in all the different domains where we want to get value out of agents, how do we teach the system and give feedback to the system of what we do want it to do? Right. And so there's a bunch of fascinating, I think, like, technical problems within there. But we're seeing this ecosystem emerge where, for example, you might have a lot of domain experts who make like a fact verifier that is one step. Like it looks at, you know, some piece of legal text and it says true or false as, you know, authenticated by a lot of different law professionals, whether some assertion is safe to make given that, right? And that's one piece that you have a lot of human expertise that can be captured. And that could become a really critical step for what Ali describes, these kind of, semi-automated pipelines where I trust my system because my system is composed of some different steps, each of which has been built and validated with a lot of domain expertise. But then I still have a final check on my, like, business decision at the end.

Definitely. And there's this really interesting and important vocabulary shift that, you know, we've been talking about and has been underlined today. In going from human in the loop to human as infrastructure. So Ali, could you just talk a little bit more and unpack what actually changes when we think about and make this conceptual move?

Yeah, I think, I think at a high level, there is, you know, there's lots of spend on compute and inference and data centers. And I think this is, I think the way to think about this, it's really a bet on future inference. And therefore there needs to be a lot of capabilities actually unlocked for that future inference to be, to sort of be there and for the data, for the compute to actually be used up. And I think to make that happen, there needs to be lots of environments where models are being trained on with human structured judgment. So we're moving on from this notion of human-in-the-loop kind of RLHF feedback where responses are given and it's sort of reactive for humans across different domains or giving feedback and in a sort of a preference labeling structure to a lot more real-world environments that resemble long trajectory tasks for models to kind of replicate. So when we say, you know, human intelligence infrastructure, what we mean is building these real-world environments that are not only resembling kind of question and answer pairs or this sort of reactive feedback by humans, but more so really distilling the entire workflow of, of an end-to-end project. So like one, one example is, um, if you, if you think about, you know, a law firm, when they deliver something to their end customer, it's, it's often some notion of maybe a contract is, is redlined, and then there's maybe some memo that comes with that redlining and a few other artifacts to, to get to that complete state of that task. There's You know, there isn't a lawyer going in there and kind of preference choosing a few things to then get to that ultimate outcome. It's more so a very complex trajectory of associates in that firm that are helping with some of the redlining. Maybe there's outside counsel that comes in and gives up some opinion on some niche part of the memo. Maybe there's a financial expert that helps out with some maybe valuation or something within the memo. There's like a bunch of different judgments that are put together to then ultimately result in that deliverable that is economically valuable. And I think the way we get models to do similar things is by replicating those real-world environments versus being kind of reactive in this RLHF sense. So our goal is to really build towards that future of sort of human data, which is each data point is these long trajectory projects, which do require multi-expert judgment versus like one kind of simpler data point that's created. I think that that is, that is simply the only way that models will be able to actually do these long trajectory tasks, which, which make them a lot more useful than, than they are now.

Absolutely. And an important, you know, underline there too in a lot of the enterprise conversations that we've been having is that if this human judgment is the infrastructure, then it needs to be reliable, scalable, and quality controlled. So Andrew, how do you, how do you view then actually building this? What does quality control look like for human expertise at, you know, the scale at which Microwon operates?

Yeah, obviously quality control gets harder to sort of quantify and measure and even be consistent about the more complex the task, right? You know, if we imagine, you know, really simple human judgments, you know, back in the day for machine learning is like, is this picture a dog or a cat, right? You know, I trust a lot of humans, although it turns out that even in ImageNet where you wanted to do this like hierarchy of visual categories, they had crowd workers. So not especially trained people on like animals and people actually confused different like subspecies of, I think it was like cats or something. So it was like some species of cat was entirely mislabeled in its image dataset. And that was, you know, that was caught later and I think it's long since been fixed. But you can imagine when we have, you know, tax accountants trying to give feedback on a multi-step workflow of an agentic system that's maybe using multiple tools, that's looking up in subsystems, even agreement between two professionals who do that might be quite low. If we frame the problem as a very simple, you know, do you and I agree on like the exact same way that we go about writing an essay? Or we're seeing this now in all of the agentic coding workflows, right? What you find is that these tools are really powerful, but they're not 100% closed systems. And so in coding agents, we have the domain expert software engineer who's sitting there and kind of working with the system, maybe not giving it feedback for training, but basically figuring out how to compose the multiple steps, right? And so I think quality control and like how I reliably inject domain expertise is one, I wanna be working with my domain experts to establish what are the sort of anchor points or the good windows into my system? For example, in these agentic tax accounting workflows that we might imagine as being pretty complex, I think there are good intermediate steps, right? We can ask about, is each individual, like, federal return for a couple correctly verified? Or have I at least filled in all the factual information before I start evaluating what deductions someone might be able to claim off of this information, right? So I think with domain experts, we can find these things that are sort of easy to verify, right? In, in RL training, we talk about verifiable rewards as being really critical. And I think with human expertise, this notion of verifiable intermediate steps. And so I'm not going to ask my domain experts to agree on exactly every little step of how we get there. But with my domain experts, I can, I can verify that multiple different accountants would produce the same intermediate artifact or intermediate representation for a task, and that when they're giving feedback, I can anchor my feedback to my agent, right? Because it might be okay to let an agentic system on its own sort of explore and learn and be robust to things. But then we have these kind of bottlenecks or choke points that we know how things should look.

Definitely. And on that note, what do you think the— and when it comes to human input, you know, where do you think the most difficult parts of human input that are really hard to replicate lay? Is it, you know, domain expertise? Is it contextual judgment? Something else that comes to mind? Just thinking through, you know, this more technical side of when we're really trying to implement these human inputs and signals, behaviors, and judgment into these systems.

So there's a weird fun story about all of this, which is a longstanding issue in machine learning is that human annotators have common sense knowledge, and they understand the broad domain, kind of, in a way that is unfair. So if you think about like looking at a little sample of like 10 by 10 pixels, it turns out it's very hard to tell what an image category is. But because humans are specially tuned to that and because maybe you've seen a lot of things like that, you might semantically infer like, oh, this is like very pixelated, but this is a cat, right? Similarly, when you're thinking about annotations for self-driving type systems, you have to be very careful that you're not inferring, oh, this person is about to change lanes. Because I, the annotator, am like watching the video and I see a bunch of contextual things because now I've created a label that is essentially impossible for the machine learning system that's only given this little bounding box picture of a car and is somehow supposed to know this like deep semantic thing about like the intent of that car. Now, what's exciting about all of this is With modern AI systems, we're just really leaning into this. We're just saying, like, let's try it because we're kind of assuming the opposite now. We're assuming that our LLM systems and our agentic things can have so much reasonable sort of like context and common sense knowledge sort of baked into them that we're putting them up to tasks that are unfair. It's like, you should know that this is a happy description of a scene versus a more somber mood. Right? Because it's sort of inferred in different ways. And so I think that that's quite exciting. But this is also a huge challenge because we don't know where the boundaries are, right? Because in many cases it can be— why this was such an issue for so many years in machine learning is it's really dangerous and unpredictable to ask my machine learning system to make predictions about like something that it doesn't have input information related to. And so now, like, we're getting away with that, and it's really compelling when we are just sort of drawing on the sort of, like, shared semantics in our, like, agentic sort of multi-step stack or what we're asking LLM to predict. But, you know, we can run into problems here. And so I think that this, again, for domain experts that are, like, working and training these systems, this is something that I think they benefit from being aware of is, like, we're asking the system maybe to infer too far because it doesn't have accurate context. And we need to think about what getting the right input information into this, like, you know, module or this type of call to an LLM.

Yeah. And I think just one quick point to add to that, I think a really good example of exactly what Andrew just said is context-specific evals within enterprise workflows. And, you know, I'll give one AI system that we have at Microwon as a sort of a concrete example here, which is our AI search system. Our AI search system is very much, very much different than others because the human experts that use it, which are our recruiters, they're using it for a workflow that doesn't really exist in typical recruitment workflows outside of our, you know, of our enterprise context. Which is when they search for finance experts or doctors or lawyers, whatever it is, the expectation is not to pick 1 or 2 people for some, you know, usual hiring mechanisms, but instead to pick like 150 for, you know, within like 3 days. Right. And the workflow and the search kind of mechanism there is wildly different than, let's, you know, let's call it a more normal recruitment search. And so when we run evals on this, on this, you know, internal AI system, we, one of the really important inputs to experts is giving them kind of self-contained context of our company and what the actual desire of the search is. Because if we have recruiters that are external that are doing, you know, evaluations on our system, they will sort of assume a typical recruitment workflow that they're, that they're used to. but in our context, again, is a, is more of a high-volume workflow as well as a very diverse set of experts that are being hired, which is very unique to the, the business that we're in. And so this, this idea of making kind of instruction guidelines that are very self-contained, that explain the business context deeply and, and having, you know, human data managers and other folks that are really making sure experts are understanding those before they actually start the evaluations. Is a very important and nuanced kind of input to have. Even if you're an expert in any domain, you need to kind of change the framing and, and the, the lens that you're looking at the given problem based on the enterprise context. So, uh, I think, you know, this is sort of one example of, um, contextual evals and, and there's, there's thousands of other examples, which, which is why it makes it, makes it very difficult for models to perform. Even if a model is really good at semantic search or kind of finding the relevant profile matches, in our context, it needs to be still altered quite heavily for it to work well.

Absolutely. All right, I'm keeping an eye on the clock here so I don't get in trouble, but we're going to open it up to some audience questions. I'm going to ask you both one last question from myself while The audience can just put some questions in the chat and we'll get to as many as we can in the time that we have. But to wrap it up on our end, I wanted to ask you both, what is the single thing that you think is most misunderstood about the relationship between human intelligence and AI capability today?

Andrew, you wanna go first?

I don't. I think people often frame it as competitive in ways that are missing the point. I think all of us are already superhuman compared to humans of 100 years ago in walking around with our smartphones and we can be creative and build on a lot of these technologies that we're starting to figure out how to tinker with. And I'm not at all worried about being replaced by AI or having bright, ambitious people with ideas replaced by AI, because I think they'll just be doing way cooler stuff and more productive.

Yeah, I agree with that. I think this notion that AI will replace humans is, I think, a huge misconception, no matter how good AI gets. I think in every domain, which I've said too many times, humans will simply operate agents and their job will just become more fun, as we've seen already in a lot of different use cases. And I think certain use cases will happen faster, like coding and recruitment and some areas of law where the sort of vector span of tasks change pretty quickly for those experts, which means the transition becomes a little bit more difficult. But I think in most, in almost all jobs, I expect the transition to be doable, essentially. And, and, and once the transition kind of iteratively happens, and it's not this like binary transition to operating agents, but, but this like very iterative one, the jobs of humans will just get more fun over time. And I think that the second thing I would say is, I think because of all the AI hype, there's this, there's often this notion that humans that distill knowledge into models The way models learn from those data points and knowledge is, is like always very emergent. And I think this is also not really true. Of course, coding is a good example where this is absolutely true. Like models learning really good coding results in a lot of capabilities that are beyond just the data that kind of gets distilled into them, which therefore means it is emergent in nature. But I think in most domains, models are actually not so emergent. They simply learn from the data that is distilled and it sort of is left at that, which I think is, I think overall that's a really good thing for humanity. Also, I'm going to grab a charger. I'll be right back.

Okay, sounds good. In the meantime, let's get into the Q&A here. So on this On this topic, Andrew, we can start with you. Do you think AI training as a job will offer some kind of stability in the future, or is it always going to be, you know, as, you know, volatile as the contracts people receive? So we train AI to do our job. Meanwhile, we, you know, maybe avoid more stable jobs. I'm curious on your take there.

I think that understanding what's going into AI and sort of what it's capable of is an increasingly important job skill for experts and people working in a lot of different domains. So the question of, you know, I know for sure that AI training today is different than what AI training will look like 2 years from now. Like, some things will be similar. Of course, there'll be a lot of similar themes, but we're in a time of change. We're exploring what's possible. We're bringing up new types of training for these reasoning and sort of inference scaling type of approaches that we're exploring a lot now. And so I don't think that, you know, AI training work, if you just take a snapshot today, is going to be, you know, stable in one sense. However, what's emerging from all of this is a lot of the excitement now in the research community are these reasoning inference scaling, like Ali's talking about. You can get these kind of really targeted applications for like different enterprises. And so that tells me that there's going to be this really exciting relationship between domain experts and systems that can do some combination of learning quickly, or being adapted and composed, right? And so it could be that your future is not like AI trainer in the way we think of it today, but it's actually using AI systems to do things in your domain and being able to accurately give them feedback or much more complicated versions of sort of multi-step, multi-node prompt-type compute graphs emerge from this. And so I think being an AI trainer, you're building a skill set that helps you evolve in this sort of AI-enabled domain expertise era. I've seen this happen over the last 5 years. I've seen people, for example, transition from healthcare expertise to healthcare data labeling expertise to actually like building full machine learning systems because they know the data side well in healthcare, right? And so I know that people in the world are doing this already. It's just now we have the opportunity to do it in this really bunch of new and interesting ways.

Definitely. Awesome. Another question that came in is, which human skills will become more valuable in an AI-driven future, and what skills might become less important? This might be one for some hot takes.

Ali, do you have any other thoughts?

Yeah, I think I think the way I would think about this is, um, like if I were to go back to university, what would I study? I think maybe is a good way to answer this. And I think what I would study is, uh, basically just math and physics. And the reason for it is any other domain is essentially some level of abstraction. On math and physics. And as AI systems get really good, it's actually hard to understand which level of abstraction you should learn. And so going all the way to the basic principles is, I think, the most useful because the rest becomes kind of a guess on where on the hierarchy of abstractions you should focus your efforts on. And I think computer science is a good example of this, which is, and I think this is the reason why which of course, Andre, you can, you can touch on this, but I think there's a reason why professors and, you know, university oftentimes don't actually teach like programming languages often. They sort of go to the basic principles of computer science and the sort of zeros and ones because it's sort of, if you teach Python, Python may become a little bit less relevant after a couple of years. I mean, as we've sort of seen now, as Andrej Karpathy says, English is the new programming language. And so picking the level of abstraction is, I think, a difficult thing. So what I would do is I would just go to like the very base, which is physics and math. Yeah, Andrew, what do you think?

I agree with that sort of like reasoning, although I instantiate that in a different way because I don't think of it as like physics and math is the thing to learn, but like the The thing that I think you really need is analytical thinking and sort of critical analysis. And in school, we often trick you into learning these things in a way that generalize, right? It's like, I don't actually want you to do, you know, discrete math proofs. You don't need to know discrete math necessarily in your whole life. But having that thought pattern of how you work through like logical mathematical proofs is hugely useful when you're analyzing any type of research paper. And I have joked around I feel very lucky in that I went through school when LLMs were not an option. And so I was like forced to develop these critical thinking skills that turn out to be hugely valuable when I'm partnering with an LLM. I worry a lot about students that develop too much trust in some of these systems that can attempt to do really complicated analysis, right? Like getting, you know, an LLM to write a whole business plan for you is really cool and it kind of works. But if you started a couple of companies, you'll be able to poke holes in it pretty fast. And so that is like, how do we get people, whether it's through learning physics and math as kind of like fundamentals, or whether it's through how we teach courses and how we think about like learning to make it so that you have the critical thinking and analysis ability to partner productively with, you know, agents that are a little wrong sometimes or a little wonky in ways that can be hard to recognize.

Awesome. Thank you both. All right. I think we have time for one more question. Okay. So this is for both of you. If we do reach a point where models exceed human performance in most cognitive tasks, does the human-as-infrastructure model survive, or does it transform into something you don't have a name for yet?

Yeah, I think, I think there's a, I think in that case, the human structure really continues to thrive. I think, I think survive is not really the right terminology and there's many reasons for it, but the one that I would point to as the main reason is the judgment of humans across pretty much every domain and at a societal level. Changes. I mean, you know, you could point at laws changing as a good example of that, which means that if we want to continue aligning models with humans, which we absolutely must and should, we need to keep distilling structured human judgment that is now different into these AI systems. Because at the end of the day, we want these models, whether there's some embodiment of them or they're digitally constrained models to help us as humans and to make us sort of thrive in our jobs and our daily lives, which means it needs to understand our desires. And our desires are very quickly changing. And so even if models are quote unquote better than humans in a lot of domains, we still need to continue distilling to them what, what that better actually means in, in, in the future. Yeah.

And I think that, um, one, there is a huge amount of new products to build, new automation, and just like a lot of value that can be created by having, for example, like AI chatbots that we can, for example, that interview people all the time so that we don't have to have human interviewers, right? We can do new things, right? And so that's not necessarily replacing anywhere because that's not having a human as infrastructure, right? That's like new things we couldn't do before. But when we say like, what's the role of domain experts? Again, you know, software engineers are kind of showing us the path a little bit, which is if you look at recent, you know, discussions about coding with LLM agents, you'll find that people are now kind of operating at a new level as a software engineer, which is they have maybe 5 to 10 different little Agentic workflows that they're composing and they're working across those instead of working across, you know, like different browser tabs in their IDE to get their work done. And so I always think of it as, you know, if Ali and I are gonna start competing businesses and he's going to offer something that has the ability to have humans in the loop in clever ways, No matter how far we're progressing on these, like, AI systems, there's almost certainly something that a human domain expert can do to help that business process or do to help that model that if I'm committing to the, like, no humans touching anything at all, I'm probably going to lose, right? And so, to me, like, the exciting thing is we're changing the boundaries, we're changing the relationship between domain experts, and that's very active right now. But I do see that role, it's just evolving all the time now.

Awesome. Great. Well, thank you to everybody who joined us today, and big thank you especially to Ali and Andrew. We are so excited to have you on the MicroOne team and so lucky to have you. So excited that this is just the beginning, and we look forward for all that's to come. So thank you, everybody. Really appreciate it, and hope everybody has a great rest of your Friday and weekend. Thanks so much.

Thank you, everyone. Thanks, Izzy.

Thanks, everyone.