Adarsh Hiremath - Future of Work in an AI Economy

Cool. So I figured I'd start off with a little bit of context on the company. So Mercur automates a lot of these very manual processes associated with placing someone at a job opportunity, right? So reviewing your resume, conducting an interview, and matching a person. And we work with a lot of these foundational model companies, as well as application layer companies at placing these really expert contractors who are driving model improvement through model post training evals and whatnot. So what I wanted to talk about is a little bit more about what we've learned from working with these companies and what the actual bottlenecks are to creating better models, creating agents that work at scale and whatnot. One of the most interesting trends that we've seen is a shift from compute bottlenecks to data bottlenecks, where these foundational models have gotten way, way smarter. And taking them to that next level of intelligence requires imbuing that intelligence in the model. And the post training techniques that we're seeing a lot of these labs use is just changing very, very significantly. Maybe a simple example is we started off with supervised fine tuning, which was essentially a prompt and completion pair. You'd do thousands of those, jam those in the model, and then the model would hopefully learn something. We then had other sorts of things, like reinforcement learning from human feedback, which typically took the form of a test, right? You know, you present the question, you have four possible answers, you pick the best one, and the model learns something. And now we're seeing this whole new category of thinking models and thinking agents and all these sorts of things where in order to improve the agent, you need to reward it really, really granularly via rubric, and reward it for going down specific reasoning chains. So this creates really, really massive demand for human data from experts, not crowd workers. And when you think of human data, and just like data labeling in general, you might be thinking of, you know, like, is this a hot dog? Is this not a hot dog? Like, is this a stop sign? Is this not a stop sign? Because that genuinely was what it was five years ago, right? But today it's much, much more. So, you know, we need the model to be better at rust, and we need like the best rust engineer to go ahead and teach the model how to do that. And finding that person is like a talent assessment problem, not a crowdsourcing problem. So Mercour works with all of these companies that are trying to make their agents better and their models better by placing the rust engineers, the Olympiad medalists, the financial analysts, the chefs, the lawyers, the engineers, all these people that are necessary to make models better. One interesting thing that we have learned is that a model is only as good as its eval. An agent is only as good as its eval. And that's also a topic that's come up here, right, with discussion of sui, bench, all these different eval frameworks and whatnot. Because in order to understand how well your agent is performing, you need to have an eval. All of modeling work just starts with evals. So one of the cool areas where Mercour is spending significant amount of time is automating the process of sourcing, vetting and placing experts at all of these different labs to make the models better. And the way that we've automated that is with our own agents, which we have our own evals for that we've hired people from our own platform for. So it all sort of comes together, but we have a pool of 700,000 experts on our platform, and these are people across all the different domains that we've talked about. A powerful search over profiles, AI interviews, GitHubs, papers and more. We're able to interview them at scale with an interview that understands their background, what's on their screen, the best question paths to go down, what the job opportunity is, which immediately filters down the people that we care about, and then we're able to place them at an opportunity. Over the next couple decades, work is really, really significantly going to change. The main barrier to applying agents and models to the whole economy is evals and improving them. And that's a talent assessment problem. And a lot of the smartest people in the world are going to be spending their time on creating these evals in the next couple decades. So Mercour is building the infrastructure to match and scale expert knowledge in the era of agents and the era of models by using a platform that can model every job, measure every skill, and optimize these matches between people, AI and work. So that's Mercur, and if this interests you in any way, we'd love to chat more.