Fireworks AI [Dev Day] Fireside Chats- with Adarsh Hiremath (co-founder, CTO- Mercor)

Merka is doing an amazing job helping a lot of companies, tapping into global talent, and it's definitely the frontier application company and thinking about model quality customization, all that good stuff. So why don't we start with model quality? How does Mirka think about model quality?

Yeah, I mean, I think model quality varies by use case in a significant way. So just for a little bit of context about Mercour, so we are building infrastructure to predict job performance better than a human can, where there's a very, very manual and validated way of assessing someone. Right. So looking at their resume, conducting an interview based on their background, matching them, seeing how well they do, and then refining those predictions, and so on and so forth. So we leverage LLMs throughout this entire process of hiring someone. And model quality varies based on the specific use case. When we think about model quality, a big thing is just like, how well are we doing relative to technical recruiters and hiring managers at finding a person for a role? And the most exciting signal for that is just true performance. Right. Once they're actually doing the job, how well are they doing? And if our models tend to align with that, we tend to think that is doing a pretty good job.

Nice, nice. And in terms of the lever you pull, the tool you use, I think you're very bullish on reinforcement learning. Could you tell us a little bit more about that? And what do you think about these kind of techniques in general?

Yeah, I mean, one thing that's super fascinating about reinforcement learning is just how powerful it's making the model, where, I mean, of course there are these established techniques like SFT and whatnot, but. But now with things like OpenAI's reinforcement fine tuning API and the amazing work that Fireworks is doing, you see that we're able to essentially create these rubrics or autograders to essentially define what success looks like at an ambiguous use case for a model doing a task. And what you can essentially do is repeatedly sample this model over and over and over again, and then grade the responses according to these rubrics based on stylistic correctness, the like informational correctness, whatever it may be. And it turns out that this technique is really, really powerful at tasks that require a lot of thinking, so to speak.

Well, that's great. So then it sounds like there's still a lot of work to do to come up with these kind of rubrics.

That's exactly right. And oftentimes these rubrics are operating at a very, very high level of specificity in a specific domain and require a Lot of detail depending on the use case and require a lot of expert human talent to do so.

And we hear a lot of hype in the community around building environments for agents to operate in for very clear signal or clear feedback for these rubrics.

Right.

For example, math and coding, there's very clear cut signal on whether the agent did the right thing or not. But for Merkur's case, it sounds like it's much more difficult to come up with a execution environment that can give you coding feedback. So how do you deal with that ambiguity in your evaluation framework?

Yeah, I think what makes the problem so fascinating, which is that predicting job performance is such a high dimensional task. There are some people who do well because they exhibit like insane amounts of charisma in an interview. There are some people who do well because they have tons of domain specific experience. Right. So the question becomes, how can you parse through like all of that noise for such a high dimensional problem to, like, leverage the models to actually be useful for this specific use case? And the way we think about it is just collecting tons of high quality performance data and training these models to look for signals that even like expert hiring managers or recruiters would have completely missed in a first screen.

Okay, so digging into that a little bit more, how do you see human actually contributing in the reinforcement learning process?

Yeah, I think it all starts with these rubrics, evals, and environments where like, these rubrics operate at a high level of specificity and there's very, very limited transfer learning associated with them. So the implication is that you need to have these, like, expert humans in all the relevant domains, creating a lot of these, these rubrics or evals for agents or whatever your RL use case is. And that's something that Mercur does a lot of work in, in finding those, like, expert humans who have knowledge in all of those kind of different domains, which is where we work with like a ton of labs to do that.

Amazing. So then, at what point should I expect to be hiring AI agents on Merka then? If the model gets good enough, I.

Think it's a little bit of a ways away. I think, like, a lot of people are in this, like, unidirectional mindset of just like, you know, we're going to operate in this future where we give AI agents tasks and then those agents, like, executed 100% of the way there, and then the humans do nothing. I think the future is much, much more bidirectional and collaborative in the sense that, like, we might give an agent, a task, and it will take it like 70% of the way there. And then you need an expert human to like go the last 30% and there's going to be huge premium on like whoever is able to find that person. And I think there's also this interesting paradoxical thing which is just that we need evals to even know when we've arrived at superintelligence. And definitionally, those evals have to be outside of model capability and it has to be a human doing it. So the future is really all of these evals and these different domains that define success for all these different models. And these models are going to start doing really, really complex things with lots of context, multi turn tool usage, many, many levels of thinking. And it's going to require a lot of human expertise to make that happen.

So that doesn't sound like you believe in AGI or like fast takeoff. Is that what I'm hearing? Or.

I mean, I think there's a lot to unpack with like AGI and like and fast takeoff and all of that sort of stuff. I mean, but I do think that the models will get very, very smart very quickly, and I think they will do a lot of economically valuable work very, very, very quickly. And I think in a world where models are able to accomplish a lot, it will create a lot of abundance and unlock tremendous human potential as well.

But you still believe I'll be hiring humans on Merka, not agents?

I think we'll be hiring both.

Right.

Like an agent might be taking us 70, 80% of the way there, but the last 30, 20% will require human and even whether to, to understand whether the model is successful or not. To eval the model, we will need a human to perform that.

Fantastic. Cool. So maybe last curveball I can throw at you is in a world, if there is a fast takeoff, let's say, how would Merka prepare for that?

I think we're going to need a lot of humans to make sure that the models are doing the right things. Yeah.

Cool. Thank you.