$1T/year evals market emerging (with Aaron, CEO of Box.com)

Very, very lucky to have Aaron, the CEO of Box, joining in our virtual series today in a conversation with the CEO of MicroOne, Ali, so that we can kind of disentangle some of the big questions that all of us really have around agents, but really where the industry is currently going. And both of you are operating probably at neck-breaking speed, and we are probably seeing such a degree of hyperacceleration in the industry. So, Aaron, look, first of all, welcome. This is an honor for us to have you here. Before we get into some of the questions that I prepared for this conversation, just share with us your journey in, you know, really 30 seconds. Imagine if you had just the elevator time to go from 0 to 10, right? And you wanted to share that story with somebody who was chatting with you. What would you say? What's been your story in the industry so far?

Yeah, sure. I'd probably, you know, punch all of the elevator buttons. We have, we have many different twists and turns to our story. But we, we started Box to make it really easy to access and share information from anywhere. And this was right at the moment when the cloud and kind of mobile was exploding. And so we built a platform to help enterprises manage their most important unstructured data. And now is a moment that I think we have sort of felt like in the back of our heads we've been waiting for, for, for, you know, almost since the founding of Box. But that took a few years in to realize the full potential. But for the first time ever, we can actually tap into all of this information in our enterprise. And this moment that we're in with AI is the first reason why we can now actually ask our, our, you know, data any kind of question. We can automate almost any workflow, or at least, you know, many parts of our most important workflows. And I think this is going to fundamentally transform how companies work. So we're unbelievably excited because AI agents need data and we help customers manage a significant portion of theirs.

Great. And Ali, imagine you are that person in the elevator, right? And then you just meet Inara for the first time and you happen to be the CEO of MicroOne and you equally went through this incredible transformation and you hear a story about agents and data. What would you say? Imagine that now you say, you know what, I'm going to go to the next 10th floor and I have my 30 seconds to tell the story.

Yeah, yeah, absolutely. Well, Aaron, it's good to be here with you. I think perhaps the most relevant topic of the year, agents and data.

The decade. The decade.

Of the decade. Exactly. So, I mean, yeah, the quick elevator pitch for us is we're the AI platform for human intelligence. We vet highly skilled people to train LLMs. And so you can sort of think of it as a data engine product that has kind of two core components. First is our AI recruiter that helps, you know, source and vet the right experts for any given pipeline. And the second is our data platform, you know, where the job actually gets done, where the data gets created, and where we have a bunch of review mechanisms and so forth built in. And I think the, you know, what I would tell Aaron in this, this, this elevator ride is, is that we both need to do our job in educating enterprises on the importance of evals and data in the enterprise workflows. I think this is, you know, it's one of my main focuses right now, which is this sort of data vertical, this sort of new data vertical of what we call contextual evaluations, which is, you know, basically evaluating all the action spaces that any given agent should have in very specific workflows within enterprises. And so, you know, that's kind of, I think that's the main limiting factor to kind of AI adoption. So sorry if you had an idea.

That's great. And, and Eric, imagine that now I'm gonna try to build from here, right? And now we've been hearing a lot about AI agents and many of these are extremely interesting demos, right? They're really great way to demonstrate the full potential of this. Now, Box has been working with incredibly interesting and, and well-renowned organizations. I would imagine some of them are also risk-averse company that you would not expect using agents. What are you— have you seen in terms of the excitement from a demo that you really wanna run and gonna showcase how agents work and a system that actually deploys the, the agent? So for us, I think it's important to see what you see and we don't see necessarily from the outside. It's kind of that, that interlinkage between the, the, the hypothesis you're trying to test with the demo and when it becomes real.

Yeah, I mean, I think the, you know, maybe the easiest way for me to think about it is in the demo environment, you basically aren't facing any of the hostile challenges around context engineering because you sort of, you know, you always know exactly what information to give the agent. You know, the person putting together the demo is like a 95th percentile expert in AI systems. And so it sort of knows how to prompt the thing. And it knows, it knows, you know, how it's going to find the information to work with and, and what tasks to automate and where the, you know, where the limitations are of the agent and how to kind of, you know, kind of tune it to the right direction. That's the demo. And the real life environment that the knowledge worker is dealing with is like two orders of magnitude more complicated because they don't really know how to give the agent context. The agent is, is dealing in like hostile SaaS terrain of systems with like different levels of access controls and different amounts of information it has access to. And, you know, some APIs are really fast, some APIs are really slow, some give you like all the raw text, some, you know, have to make you kind of parse and understand the information you're getting back. And, and that is going to be the story of agent I think adoption, you know, for the next couple of years, which is the companies that, you know, understand how to set up their organization, set up their IT systems, set up their workflows effectively in service of agents. And this might feel slightly disappointing to people that thought like AGI was just going to make everything really easy. But the companies and teams that are willing to change the work and change the workflow to make agents effective, they'll be fully unleashed and they will get, you know, 2x and 5x levels of productivity. And the companies that are going to wait out, you know, sort of when the model can just do everything, they actually might be fine in a couple of years, but they'll sort of miss the, you know, the early returns that are, you know, already starting to happen. And you can see this in engineering. So the, you know, engineering being maybe the area where we have the best examples of like autonomous long-running agents that are working, the developers that want to stick to exactly the way they've always built things are going to get like 20% and 30% productivity gains from AI. You know, that you're using AI as a thing that you just ask a question to, or maybe it kind of, you know, is typing ahead some of your code and kind of auto-predicting what you should write. But you're not sort of really changing your workflow to get the full gains of an AI agent that you just, you know, send a task to and it goes and does, you know, an hour of work and comes back and with some kind of product. And that, again, that's the analogy I would use in knowledge work is if you don't change your workflow, then you're going to be stuck at these kind of 10 or 20 or 30% levels of gains. And so the big question the next couple of years is how will companies, how will teams, you know, sort of re-engineer the way they work to, to be able to get these gains, uh, from, from AI and AI agents. And that, that tends to be the conversation that we have with our customers is, is that, that's actually how you're gonna get the real upside from AI. Um, and doesn't mean you have to wholesale change your whole organization. I, I'd focus, focus on the areas where, uh, there's just like meaningful upside if you did re-engineer that workflow because you could produce way more than you could have before. You could, you know, find risks that you couldn't have before. You could lower, you know, some cost in some kind of core operational bottleneck to the point where then you can speed things up dramatically. Um, but you have to be, you know, pretty targeted, uh, if you're going to go and do that kind of workflow re-engineering. Yeah.

And I think, uh, one thing I would add to that is, you know, an example that, that I would say is, uh, typical product workflows where you, when you build a product, you obviously have design, engineering, you know, QA testing and so forth. And I think one example of like a workflow that needs to actually quite fundamentally change for the age of agents is the QA workflow where, you know, for probabilistic softwares and, you know, in other words, agents, you actually can't have the existing QA processes or existing kind of QA engineering processes anymore. You, you're not looking at a software on whether it works or not. It, it is not binary by, by any means. You, you have to measure how well it works across all the different functions that you, that you care about. And so, you know, that, that's something that, you know, that, that's also what we, what we tell our customers, which is you need to kind of rethink the product build-out workflow and, and you need to look at evals as like almost as core as, as engineering, right? And I think that the example that I would use in terms of like when you actually implement an agent, you have to— an example of like a workflow that you have to fundamentally change in our case is recruitment. We're, you know, obviously we're very familiar with the recruitment agents. We— when we saw the most output and benefits from the agent that we've built, which is the AI recruitment agent Zara, is when we fundamentally changed our process internally to no longer doing interviews. Where our recruiters basically never did an interview again. And they had, you know, Zara kind of conduct all of those interviews and their function as a recruiter actually pretty fundamentally changed. They actually were able to kind of focus on the more kind of creative parts of their job that a human is, is good at. And in fact, they, they came up with kind of net new functions in recruitment that recruiters usually don't do. Like they almost became like recruitment marketing experts in a way. They were like setting up like really cool sourcing campaigns. And recruiters don't have time for this because they have to, you know, do back-to-back interviews and have to do kind of like cold outbound to candidates. And when those things get solved, the human creativity kind of flourishes in any given function and the recruitment function becomes really powerful. And so, so, so, so the way, the way that we kind of implement this across all domains and functions in the economy is is kind of embracing the, the very large changes that do have to happen in workflows, which will make humans' jobs a lot more fun.

All right. Ali, can I ask you a follow-up on this? Right. So when I was hearing about workflows, I was thinking that at MicroOne, a lot of the work we do is almost reinventing workflows because we have this understanding of granularity of the data. Right. And look, the way I come across MicroOne and the way we met is that I was doing something else in my life. I became one of the experts from MicroOne and I was surrounded by a community of experts. But one of the question is, don't we already have a lot of expertise in the data? Don't we see the data nonetheless already operating on the peak? So what is different in what we need moving forward? And I'm trying to ask you both in terms of how you're running, I think the vision for MicroOne moving forward, but how that vision will equally support Aaron view that we want to kind of, you know, rethink workflows, but we're not interested in the 20, 30%. That is great. It's, it's, it's a low-hanging fruit, but the real game changer will be when we redesign work, labor, productivity, not for 20, 30%, but it'll become— and I wouldn't say exponential, but it would be a radical transformation with this right now. So how do you see that kind of— I find it to be a bit of, of, um, not always, always, always intuitive, so be that counterintuitive about it, on expertise level and peaks and what we see, what we need to move forward.

Yeah, it's a good question. Yeah, I think, you know, the question is like, don't we already have lawyers and doctors and finance experts creating these very complex datasets? Kind of where do we go from here? Like, is this kind of the peak? Because there's obviously a lot of data we're providing to foundational models and enterprises and so forth. And if they've already trained on that data, you know, is there more to do? Like, where can we go? Like, do we have to hire kind of like Nobel Prize winners and so forth? I actually think that the way you scale, the kind of the scaling path on data here is actually not making it more expert level. I mean, again, it sort of is already maximum. Hopefully we get some Nobel Prize winners in talent pool soon, but it's already kind of maximized with doctors and lawyers and so forth. The way you continue scaling here and improving models and enterprise use cases is you create tasks that are actually long horizon and, and that are actual tasks, not, not questions and answers. The truth is right now, you know, the model providers we, we work with, a lot of the expert-level data that's created is very complex questions that require very complex answers. And they're, you know, oftentimes essays and memos or how, however you wanna kind of, kind of frame it. But, but, but they're not actually doing anything, right? It's not a, they're just answering a question. Now to do something, it's sort of equivalent to saying the model has to answer many questions, maybe for its own, and then ultimately kind of do an ultimate answer or create an ultimate file or whatever it is. And each of those answers, each of those steps have some error rate. Even if you assume the error rate is quite low, maybe 90% accuracy on average, if you take something that's 20 steps and you do, 0.9 to the power of 20, you're actually gonna get very close to 0. You're gonna get basically 0% accuracy. And that's obviously a horrible outcome. So the way you scale and the way that data continues to be really, really useful is you make the task longer and longer horizon. And so you keep the experts, doctors, lawyers, et cetera, but you make the data structures much longer horizon and multi-step. And that's where models are gonna be able to actually do things in workflows such as Mox.

So if I understand correctly, and that's actually your next question, because it's the best question for Aaron, right? The, so like the, the longer horizon, Ali, will generate probably more context to the data itself and are in the context of an agent, right? How do you build that context? Because one of the things we found out is that we think of a workflow and maybe we think as an engineering design, But then when you run the workflow, there's a lot of, let's say, gray areas that are defined in the contextual understanding and intelligence of the organizations. And I remember when in 2024, 2025, we were thinking of agents and we were thinking about accelerating towards AGI. And then we realized that with agent, human oversight was equally increasing. They were trying to offset, right, the lack of context. How do you see this in the nature of operating in the prime space in agents, right? How do you build that context with agents?

Yeah, so we're, you know, we have a natural bias to the context that is already sitting inside of your enterprise information. And while I, you know, submit that it's not all, you know, the necessary context to make agents powerful, it's actually a lot of some of the most important things. So if you go to an enterprise and you say, you know, where, you know, where are some of the best best insights captured for a sales rep to be able to sell to a new customer, it's going to be in, you know, the meeting notes with past customers, it's going to be in the sales materials, it's going to be in the training materials that somebody has done time to, you know, spent time to curate and then create and go and publish to everybody else. And so where is that all sitting? Well, it's sitting inside of documents and notes and presentations and that's all unstructured data and enterprise content. So, so you can kind of play that out to lawyers and doctors and, and, you know, research scientists, all of that data that you work with to do, to do your job. And so our, what we're working on building is, is, is really creating a system where you can have agents that can tap into that unstructured data as the critical context to do work. And I think to Ali's point, probably the first 3 years of the ChatGPT AI wave was, you ask a question, you get an answer. And that is obviously incredibly powerful, but it sort of has like a pretty finite amount of sort of work that that can possibly compress because it can only compress the things that are just like, get a fact or get some kind of decision, which is again, very useful. But it's— that's a small percentage of the work that we do. We're often having to review lots of stuff and, and collate it or reproduce it or transform it or take a bunch of information and generate all new sets of data in the form of a research report or a filing or a new contract or a memo. And so, and so the question is, how do you get the right context to an agent to do those much harder, harder tasks? Right. I want you to go in and generate a contract for me, you know, for this new client with the following clauses. And then the question is, well, what, which contract template should the agent use? Should, should and should it look at like 30 examples and, you know, find the average of all of them? Should it just do the most recent one and sort of copy that? These are the things that actually humans, you know, we kind of have a lot of that context for free because we, we sort of can just say, you know, hey, hey, hey, you know, hey Tim, what did you do last on the last contract? And then you get the answer. The agent doesn't know, you know, the answer inherently to that. It has to go and kind of self-discover where should it find that answer. And if you have an agent ask you a question every single time you want to go do a task, then you're not going to get the efficiency gains because you'll be like, well, this is sort of now, now it seems like you're not actually able to do much for me. So, so somehow you have to find this balance of the agent has to be able to find the information that it needs. It has to be able to, acquire the context somehow and then it has to make sure that it knows how to work with your data and your tools and your file system and all of your information. So that's the problem that we're working on. And I think to Ali's earlier point, evals and ground truth and, you know, this is going to be a very important topic for the enterprise because we don't have— most enterprise workflows don't have the equivalent of evals for the workflow. Like we never had to think about that. That was just like not a thing. And so you might be in a world where every workflow in the enterprise has to have evals associated with it that you can use to understand how your agents are performing. And before you, you know, you're really able to actually like deploy anything at real scale in an enterprise. Our version of this is we have a feature called Box Extract where we extract data from documents and to be effective, we have to get the ground truth data for, you know, some sample of documents, you know, as we, as we kind of start that process, that becomes our eval effectively, you know, that we use for the agent and that, that's how we kind of know, you know, are we getting better or, you know, where is it trending with the accuracy level, but that is going to be needed for everything in your business that you want to go and automate. And And maybe the only final point is, you know, we've been, we've been riding on, I think, a lot of the maybe like the easy mode of, of AI automation, which will, you know, maybe sound a little counterintuitive with coding. Coding has been the easiest problem to go and automate because, you know, the, the, there's like not like an eval really that, that like anybody, you know, commonly requires or agrees on. It's like just did the code run? Like, did it produce the thing? And, you know, some people will look at the code and be like, wow, that was really like verbose and like, like it's really ugly code. But like, the thing worked or it didn't work. You don't get that. You don't get that affordance when you're generating a contract. You don't get that affordance when you are coming up with a new, you know, kind of medical diagnosis. You're not afforded that when you are doing an SEC filing. So, so once you get to that kind of work where it is just like the thing produced it exactly the way that the government will, will, will accept it, did the thing produce it in a way where you're not going to get disbarred because it sort of found some, you know, you know, fake citation. That's actually the much harder form of agentic work. And, and that is going to be obviously the next, you know, you know, 5, 10 years of, of work automation. But that's going to require a tremendous amount of evals. To be able to actually see how we're progressing against that trajectory.

Ali, can I ask you a question on this? When Arun talks about eval and you try to translate it in terms of data, right? How do you see this in the nature of what we do at MicroOne? And I'm thinking in terms of, I think we have been expanding on the data training quite remarkably. We also might imagine that there will be some verticals that will become dominant for agents, not necessarily in a deterministic way, but probably as a transition. 'Cause I would imagine that some of the conversation on evals are kind of backstage of the quality of data that we have to evaluate an agent, right? So how do you see this in the line of MicroOne in terms of how much diversification you had on the dataset right now? And where do you see eventually this becoming emerging or kind of emerging landscapes of data that will be reliable enough for that kind of evals and growth that Aaron is talking about to eventually accelerate so that we don't necessarily operate always in terms of not just transactionalism or incrementalism, but we're really thinking about a leapfrog.

Yeah. So the way we look at it is I think there's sort of like 3 main data verticals like that I would sort of categorize it as first kind of frontier evals, which is you are trying to get, I think, the core truth of certain domains, medical, finance, legal, all the ones that, that, that are constantly mentioned. But also there's a longer tail of hundreds of other domains that that the foundational model companies are using to train their model. And I think, you know, there's an important realization there, which is, you know, fortunately, but also sort of unfortunately, like the way you build general intelligence is actually, you sort of have to train on every domain. And of course, there's emerging capabilities in some domains like coding and a couple others, but there actually isn't emerging capabilities in most domains. And so if you want to build a generally intelligent model, you have to build environments for all of those domains and the many subdomains that they have. So, so this is what I would categorize as, as frontier evals. And I think the, the expansion here is massive. You know, the, the, the, there's a, to, to unlock these new use cases, the, the labs must basically spend in all of these different domains. The, the second is kind of what we, what we've been talking about in terms of like enterprise workflows, which I would categorize as contextual evals, which is you, you still need domain expertise, but, but you have to really kind of hone in on the very specific ways that any given enterprise is connecting their admin portal tools and the retrieval between those APIs and so forth. And that will require very context-specific evaluations. And enterprises will soon change their workflows to have, like Aaron was saying, an evals framework for every single function. And then lastly, there's, you know, there's a real-world robotics data, which is not really a thing yet, but it's going to be very soon. And first, it will be kind of a large pre-training set created to, train physically intelligent models in very basic ways. And then ultimately, we'll get to the kind of expert-level data in robotics as well, which will be experts doing things in the physical world to train robots. So I think these are kind of like the 3 broad categories, but to add on to Aaron's point earlier in terms of like every function in a company will require evals, you know, I sort of, the way I extrapolate that is like every function in the economy will actually require evals. And there's, and there's, there's an argument to be made where a, you know, let's take a lawyer for example. A lawyer currently at Microwon gets paid roughly 25% more than their day job to create structured data for models versus unstructured data for their law firm. So the reason why they're getting 25% more is because the, you know, the economy as a whole has decided that, that, that structured data from that lawyer is actually more valuable than the, unstructured work they do at their law firm. Of course, you cannot have one or the other. The economy must continue functioning. But, but, but there's some percentage of, of, of the overall work that every function, every— all functions in the economy do that will eventually go towards structured data creation and basically structured human judgment. So if you assume that this is true, and of course there's so many examples of it already, then you must assume that some material percentage of the overall human labor spend will actually go towards human data and more generally structured human judgment. And so we, you know, we look at this as, you know, the phrase we like to use at Microwan is the future of AI actually requires human brilliance more than any technology ever. And this comes in many ways, but one of the most important ways is the structured judgments.

We, we, we've had, um, uh, you know, to this point we've had customers— so as I mentioned, you know, this, this product capability where you can, um, say give us all your contracts or give us your invoices or your research documents and then we'll extract the, the structured data from that, that, that set of documents, uh, we'll have customers for the— that actually for the first time ever have to go in and, uh, and, and apply, you know, effectively human resources at labeling their, their data because you have to generate the ground truth if you're going to go and automate the rest of the workflow. And so in this very interesting fashion, a lot of people, for instance, have been concerned with, well, what happens to the human work in a lot of these use cases? We actually have use cases where it's inventing human work because we were never looking at that data previously. It was actually just going off into some archive system that no one ever paid attention to. And so now, for the first time ever, we can make that data valuable and it can produce even more value for the organization. And oh, by the way, ironically, it requires, you know, human work and human judgment to, you know, set up the thing in the first place because you need the eval and the actual, you know, ground truth produced. And so, so there's going to be like all new ways that we have to, you know, kind of use our, you know, human intellect to be able to create these systems to allow us to actually automate anything.

Yeah. And I think the, the, sorry, Mark, just, just one more point here.

Yeah, go ahead.

Go ahead. I think that the invention of, of new work is like, you know, incredibly important point because there's obviously the, the AI training aspect of it, which is this new huge job sector that is now hundreds of thousands of people working on it. But, but, but I think even more interesting than that is the freedom that humans get when certain parts of their job are the tedious parts of the job automated, they get to actually invent the net new functions, which of course recruitment was an example we used earlier of our recruiters inventing these new functions. And, and so, and then, and then the cycle continues of those functions being automated with their structured human judgment, and then they go and create higher level functions within their, you know, domain of expertise. And, and so I think this notion of, we're working to solve the last mile just doesn't exist. There is no last mile. We will continue to go forward and there will always be another mile and the mile just continues to get more fun for humans.

Yeah, I think there's some kind of like, you know, asymptotic thing of like you never reach the end of the last mile. It becomes the last, you know, 100 yards and then the last 100 feet and then the last, you know, 10 feet. And we will just be doing this for, you know, we'll have this webinar in 50 years from now and we'll be talking about the next last mile.

Exactly.

Look, I think, guys, look, while we were talking, we've got tons of different questions coming in and Sydney's helping me to filter this. Maybe I do the last dialogue between the two of you and then we can take some of the questions from the audience. We have a pretty large number of people who are watching us right now. It was kind of inspired on a late night riffing that you were having Rabbinale on the, the Jevons paradox about, sure, we're gonna make this more and more efficient, but as it becomes more efficient, eventually it'll become even more consumed in many ways, right? And it goes to the point we'll never get to the end of it. This is an ongoing, never-ending journey. And I remember in a different conversation you were saying, look, even when we're gonna be able to train on everything we know, we'll keep on changing 'cause humans change, right? How do you see this somehow get into a point of balance? And because I think the industry is going on, on hype and to some degree on steroids for a few years now where we have this idea that we are accelerating and we're going to get to a certain level of performance. But even when we get there, we realize, as you just said now, right, there is no golden mile. It's a golden mile in perpetuity. So where do you find that the system, while it keeps on being, you know, an evolution of some sort, We'll find some form of balance where we'll mitigate some of the adversarial input that might come from, you know, the way the industry is currently training the model. We'll get to the point where we're not going to be too concerned about the legal implications of the bad decisions. We're going to get to a point where we train the data, but what we have is already quite good for a threshold that allows us to be functional with the technology. So where do you see this, this balance or disequilibrium point happening?

Yeah, so Aaron actually wrote an amazing piece about this. I will pass it on to Aaron to answer this mainly, but the only thing I'll say here is, you know, we often get the question of what happens when synthetic data works really well. And so, you know, the sort of Jevons paradox in our case is this like notion of synthetic data working very well for evaluation. And so forth, my answer is that this is actually the best thing that can possibly happen to us. If, because if, if you look at the current pipelines or like any kind of training pipeline, whether it's enterprise, AI labs, et cetera, they are, they, they are taking their ground truth with human data and they are multiplying it by orders of magnitude and, and creating many new variations with synthetic data in almost every case. Of course, this is, you know, some design pipelines don't have this, but, but in almost every case, this, this is, this is true. And as that continues and as you are able to multiply that even further with more and more variations based on the ground truth human data, each human data point becomes way more valuable. And as each human data point becomes more valuable, we, you know, we won't just consume, we won't just stop consuming it. We want to consume a lot more of it. And in the human data case, we will have, you know, we have practically an infinite amount environments to consume data in, and we will be able to explore that infinity a lot faster versus the current very finite set of domains that we're kind of training on as human data becomes more valuable. But yeah, I mean, I'll let Aaron, you know, I think Aaron's article was great on this.

Sure. Yeah. I mean, the thing that I get really excited by, and I've just witnessed enough tech trends to catch the kind of maybe cognitive dissonance or juxtaposition that tends to happen. I remember, you know, when cloud, you know, first came out, the question was, do you need IT people in a world of cloud computing? And it turned out that actually what cloud computing did was it made software and cloud cloud services 100 times more abundant. And when you did that, you actually needed even more IT support because we were digitizing everything. When everybody had thought actually cloud was going to automate all IT problems away and you wouldn't even need IT people as a result of that. That was sort of one thing. There was a period when, you know, everybody, when like Uber came out, everybody was like, well, okay, all of a sudden now the taxi drivers go away. And when— and actually what you should have realized was this is going to 100x the number of, you know, people that are going to be, you know, driving in the future. And so my tendency is to look at problems through the lens of if you automate a certain task, would the world want more of that task done or less of that, or like, or the same amount, you know, in a way that was sort of disproportionate to the cost savings of you know, the task that you automated. And I would say most forms of where we've seen AI agents, if you, if you can automate writing code, the world will just want 10 or 100 times more code. And, you know, short of, of some very amorphous AGI definition, if you wrote 100 times more code, engineers absolutely don't go away in that world. Because now you have all of these new systems that have to be run and maintained and managed and updated. And so the more code we generate, counterintuitively, continues to make all engineers incredibly powerful and useful in the future because every line of code you write and that's in production is eventually something that's going to have to be updated, maintained, you know, deprecated. And that you will still need lots of engineering to go and do that. And so then those are the easy examples. But the exciting part is actually when you can start to deploy AI at problems that we never actually had time to spend on before, we never could have automated before. And then what does that now open up for the first time in that organization? You know, if you could read every contract in your entire enterprise and ask it any question, would you be able to open up new insights that would lead to new, new sales? If you could look at every single, you know, research paper ever written in any field and you're in a particular domain, would you, would you be able to progress through medical discoveries faster? All of those things, by lowering the cost of one aspect of work, the demand goes up for all of the surrounding areas of work that that thing touches because we were previously bottlenecked by that particular function before. And so And so you're just going to see this everywhere in the economy is if we can do something far cheaper, we will do way more of it. And when we do way more of it, people will still be necessary to do that thing. And so demand will go up for the people that are actually helping support those functions. And that does not appear to be slowing down in almost any category that we've seen AI really go in and attack.

Yeah.

So let's look at that. That's fascinating. And It's amazing how like 40 minutes have already gone just in conversation that I found at the early stage of my outline of questions. So if that's okay with you, maybe we are bringing on board some of the questions coming. I won't be able to go through all of them, but maybe I'll try to consolidate a couple. So there's a couple from Rama, and Rama, I won't be able to maybe ask both of them, but maybe I'll ask the second one, which I think is interesting for us in terms of where also we're going to bridge it to the next webinar next week. Um, so Raman's asking, and Aaron, this is a question that I love to ask you, right? Uh, so if you're running a multi-agent system, take in finance, and you know there is the presence of predatory algorithms that are, are likely going to try to leak data to, to undermine the safety of the system. So how have you been reconciling with the fact that agents are not only one-sided, right? But there's also another side that could be adversarial. And that can create a security hazard for a client that might be already risk-averse, as it was mentioned before. So how did you navigate that, the space in your line of work?

Yeah, I mean, this is, this is where some of the debate around, you know, kind of AI will sort of, you know, kind of make all existing software and systems irrelevant. I think this question gets at the point of of effectively why that will be impossible. You need to have some relationship between deterministic systems and non-deterministic systems where, you know, the agent— agents are really going to want to perform whatever task you give them. And they very happily and very gladly want to answer any question you have and want to leak any piece of data that you want them to leak. And they can be used for good or or bad, and they will— any system they have access to, they will traverse that system and find exactly what you're looking for. And so— and it's just like, it's hard to mitigate that. You can't, like, prompt your way through that. Like, that is just the nature of these systems. You will always be able to override the system prompt. You will— like, it's just we can't solve that problem. So if information can enter the context window, then it can get out to somebody else. So that's our general sort of mental model for how to think about AI security, which means then you need a lot of systems— not a lot, sorry. You need systems that actually then are responsible for ensuring only the right agent has access to a particular resource or a particular type of data. You are figuring out, is that a trusted agent? Is the user on the other end of that agent trusted? Should that agent actually have access to that underlying resource? Are you doing things like scanning for the, the question that's coming in or the question that, or the answer that's going out. Do you have an understanding of the classification of the data that, that it's interacting with? All of those things we actually did before AI existed because humans had to have those exact same requirements. And now agents have to ride along that, that same set of guardrails and be able to adhere to that same set of principles. Excuse me. So, so I think that the fundamentals of, of like of just product security, data security don't go away. And probably if anything, the magnitude of the severity of getting all of those decisions and architectures right have gone up by an order that, you know, have gone up by orders of magnitude because in a world of 100 times more agents, then your risk is 100x bigger than it was before humans. And so you have to design really secure systems that you don't have data being accessed by the wrong untrusted agents that are able to access or leak data to the wrong person.

Yeah, the reason why I started with this question is that next week two of our researchers are going to be on webinar and we're going to talk about the paper they have written about adversarial prompting and red teaming. So I think it was just a great way to also— Awesome. Remind that we're going to do that. The other question is actually an anonymous attendee's, but I found the question is interesting for like kind of a bridge from what Aaron said, right? Regardless of the set of conversations you need to have at the level of the agent, there are conversations about the level of the data that needs to be truthful to the point that you can trust that the data is coming from a solid point. How did you reconcile this challenge? Because you can say maybe at first was adversarial prompting. It was just training the data. But how do you get to a point where you can say this vertical is built on extremely reliable data? What makes it truthful other than expert in that case?

Yeah, this is a really hard problem, obviously, trying to kind of create a structure for how to get to the truth in every domain. I think the infrastructure we built, all the products that we built at Microwon kind of strive towards this aim. It starts with vetting the expertise of someone and first checking if they're truthful about the skills they say they have because they will train a model on those skills. So, you know, broadly, that's kind of the first step of it. The second is, you know, creating kind of a mirroring similar to research peer review, peer review mechanisms where a data point is created, you know, As you know very well, Mark, that each of these data points often now take 30, 40, sometimes even 50 hours to create one data point. And so it's almost like you're writing a paper in a way, right? And so, so you have to have like a peer reviewer on that quote unquote paper. So, so our data platform has a bunch of mechanisms built around expert peer reviewers. And, and then ultimately there's, you know, for, for very complex areas and for, for areas where the model's already actually doing a very good job and we need to get it to do even a better job, we actually have to have multi-review processes where there's kind of a tiebreaker reviewer that comes in and ultimately decides what is the answer based on the context of the kind of two experts that, that ended up working on something. But I think, you know, the last thing I'll say here is the way you get to truth is you have to actually replicate real-world environments as much as you can. The, the, the what happens in the real world and, you know, when, when a lawyer does some sort of M&A and when a lawyer, you know, kind of delivers some, some redlining to a customer or finance experts delivers an LBO model to their enterprise customer, that is sort of the truth, or at least is a proxy to the truth. The customer is paying for that. And so we have to try to replicate that as much as we can. And part of it is, part of it is, you know, creating very realistic tasks for models to fail on. But also another part of it is having multiple experts work on certain tasks where it almost becomes like a project that, that that resembles the real world where, where multiple experts collaborate on that project before the delivery happens versus like one singular expert creating the data point. So, so we have to kind of, you know, kind of strive towards getting tasks to be as real world as possible and therefore as truthful as possible. Yeah.

Jens, look, usually we try to stretch this conversation a bit more, but this time I'll try to stick it to our stipulated 45 minutes because I know it's, it's important that we do that, especially on a Friday. The fact that we could— I couldn't even start the full flow of the question. This shows you that probably we need to have a comeback with more time for the question, because I think you are unleashing so many questions. And I think the fact that people are so engaged is that we still need a lot of clarity as we move forward. And hopefully, we can talk and see when maybe we can have another session. We can go deeper with this, because this community really needs people like you and Ali to help them, at least Just to help them, I would say, formulate their next questions rather than necessarily looking for quick answers. I want to say thank you for both of you for having spent time carving even like close to an hour away from the line of business. I know it's not easy, but I appreciate that you guys were able to do this on a Friday. And I have enjoyed the conversation that we will have in another meeting between the two of you. The only regret is we were not physically in a place where we could drink something together, but we We can always fix those problems, right? Eventually we can have a bit of a salon, right? Where we can have some drinks and a conversation on a camera and then it becomes a live event. But Aaron, first of all, thank you so much for having spent time with us and the entire MicroKwon community is extremely, extremely humbled that you decided to spend some time with us. And the success of Box is something we look up and we aspire to and it's amazing that you could share so, so openly your journey. Ali, it's always great to see you on the series and make sure that you are, you know, kind of equally sharing your own journey, right? As an entrepreneur who has created an incredible success story for all of us. For those of you who are part of the community of experts, please do not stop the conversation. There is an expert Slack channel where you can keep on animating the question. We'll try to look into that too as much as we can. This will be recorded. So for people that want to have another view on the conversations, there will be a YouTube link. But guys, thanks so much. It's been a pleasure to see you on this and I wish you a great weekend.

Awesome. Thank you.

Thank you guys so much.

Cheers.

Thank you.