Waleed Atallah (Makora) x Dylan Patel | Researcher Conversations at GTC

All right, we're here at GTC, Electrifying Energy. All day there's incredible announcements, people to meet, and all night, what did you do all night, Waleed?

I stayed home. I was whipping with 3 different Claude code sessions. No, no, no, I'm kidding. Of course, obviously a bunch of fun stuff happening.

Well, so Waleed, you're here. Tell me about your company. What are you working on?

Yeah. So we're Makora, formerly Mako, now we're Makora.

So you had to change the name, but you also changed what you're working on. A little bit about that.

So basically it's moving up the stack. So what we had been doing previously was a lot of work on improving quality of kernel generation and doing a lot of work on validating kernels that you do generate. These days you can actually get pretty nice kernels out of Cloud Code and Codex and everything. And I think the entire Silicon Valley, like 80% of the startups here, are starting to rethink everything now that coding has come this far. I remember when Sonnet came out, like Sonnet 3, for when we first saw models that were actually really good. And I remember people were talking about, they're saying, look, all right, it does frontend, it does this, but does it do the complicated stuff? Does it do kernels? Does it do compilers? And it turns out the answer is yes.

I think the funnest thing about the Opus 4.6 announcement was like, make a C compiler. And then it spent like $20K and it made a second bug. It's like, wow.

No, and you can like nitpick and be like, oh, well, there's this bug, there's that bug, there's this bug. But I mean, come on. Like you kind of just like told it what to do. So what we've been doing at Makora is, you know, it kind of forces you to get into like the product aspect. Like what do the people whose job it is to, you know, make things run fast? What does that person do throughout their day? And how does it change now with cloud code? And how can you build a platform that makes their life easier.

So before stepping into what you're doing now, I actually want to ask a bit more about the CodeGen side, right? Everyone's so excited about CodeGen. It's been something that people have been investing in for some time. There's numerous startups now in this space. Why are you moving out of it?

Well, I actually don't think we're not moving out of it is the thing. So we still, a core part of this is that We use LLMs to write high-performance, low-level code. And what we've done is the platform that we're building, we want Claude to get better. We want Codex to get better. We should use that as a tailwind. And then how do you build the rest of a platform around this kind of a capability that you can just keep assuming is going to get better? The point is that you cannot have code generation itself be your product. I think we're going to see a lot of basically Giving the, taking the codegen capability and then making these kind of end-to-end platforms that just do a lot of the work for you.

And what goes into this end-to-end platform then? Is it, you're making a bunch of validation engines or what exactly are you doing?

That's definitely part of it. And that's a really important part because again, you have to have that feedback, but the kernel codegen that we started with input PyTorch, get kernel. I think the next step is going to be input a Hugging Face link. Get full VLLM deployment with whatever kernels you need in there. And then like, let's think about InferenceX for a second. Well, I think the obvious next step for what's probably happening with you guys, you see this Pareto curve where there's a bunch of different curves. Now the question for everybody else is, wait, I want the good curve. How do I get that? How do I get that for my model, my use case, which is maybe not the exact one that's on here? And so I think a big one is going to be, here are the GPUs that I have, or here is the cloud access I have. Here's the model I want. Here are the SLAs that my customers expect. What's the best way to deploy this thing. And that's going to need a lot of testing, a lot of infrastructure, a lot of hardware to figure out what that Pareto curve is for that specific use case.

That's actually something that we're doing in InferentX, right?

It's the natural next step. I would expect that. Yeah.

At least initially we were 3 models because we were trying to implement things like evals and implement things like YDP, disaggregated pre-filled decode. We're implementing all these features. But over the last month, roughly, we've added, we've tripled, I think we've tripled the number of models or doubled the number of models. And those have been like day zero, we got the Pareto curve basically just by telling Claude to generate all the different configs, right? Pretty simple. And then cut all the configs that make zero sense to run, right? Like you're not going to run PP8. You're not going to pipeline it 8 times or you're not going to do certain configs. They just don't make any sense. So then getting it to, from first principles, just eliminate a lot of the configs. And then there's configs that you actually don't know the answer because like, well, this one theoretically is better from a roofline basis, but maybe the kernels aren't optimized there yet or whatever.

Yeah.

And then we run those and then we have the Pareto curve without using too much compute and able to implement a new model, at least in the current leading VLLab, SDLang. It's still a very manual engineer-in-the-loop process, but I can I can see what you mean, like that's going to be something that's much sooner to have.

Yeah. And I mean, I feel like every 2 weeks, like the limit of what is being automated changes, right? So I totally think that everything is going to end up being automated at some point, right? So what's next for InferenceX? I'm curious. Are you guys going to actually do the dynamic submit a model and it all runs?

I mean, generally we accept PRs. So if someone wants to submit a PR for a model, great. The goal is to do day zero support of models now.

Now what's with GPU? Spend on that? Or just capacity over it?

Thankfully, we have great partners that give us free GPUs, including CoreWeave, Microsoft, Oracle, Nebius, Crusoe, TensorWave, AMD, NVIDIA. And I'm going to kill my— there's one I'm missing. I know I'm missing one. But we have great partners who give us compute for free because this is a free open source project. We actually make money off of InferenceX. Obviously there's a lot of learnings that we get that we then turn into other products. But InferenceX, I mean, we're going to have day zero support for all the new models. We're adding more and more evals because it turns out people's kernels cheat.

Yeah.

Right?

That's evals, man. I'm telling you, validation engineering.

These things are faster, but it's like we only implemented GSM8K so far, but even in GSM8K, you see a 10% swing even on different points on a curve for the same hardware.

Really?

Because something they did on the curve, something they did on the kernels. Something I did somewhere.

Oh, different kernels. I thought you meant like the full exact same, but just—

Yeah, it's, it's, it's along the sweep, right? So when you sweep everything, like the kernels for this optimization point, that's great at throughput or this one for latency actually has different performance.

Yeah.

Um, which, which is sort of like, makes me wonder, like, you know, people are always like, oh, OpenAI downgraded GPT. Oh, Anthropic downgraded GPT. Oftentimes it's in there and they're like, it's because They've gotten so much more traffic and it's like, well, actually perhaps there's some kernel screwery here.

Yeah. Do you guys use open source models?

No. Why would you use open source models? Everything on InferenceX is open source. We love open source models, but like, stop being poor. Just pay Claude.

Yeah.

Pay Anthropic. Pay Daddy.

No, we've done a bunch of RL on some of the open source. We've done it on GPT-5 too. But yeah, the closed source models are a step ahead.

Yeah, exactly. It's like 3 months difference in model timing or 6 months difference in model capabilities ends up being a year or more in terms of when you're able to release a project and then get the traction and get the customer feedback and this iterative cycle. So it's like, and this is why we spent, everyone, I encourage everyone to use fast mode, even though it breaks the bank. It's like, use fast mode.

Yeah, more tokens per second is more. More intelligence.

Yeah, more intelligence per second. More intelligence per second is more revenue eventually, even though it's mostly just a cost.

You're sounding like Jensen, man. Buy more to save more.

Well, actually he sounds like me. Oh, come on. No, I'm just kidding.

Of course.

Did you see the keynote? I'm just saying, I'm just saying. I saw the keynote.

It was good.

Dude, he spent 5 minutes on our slide, said my name. He only said 2 people's names in the entire keynote, mine and the Open Claw bro. And now the Open Claw bro deserves it. I don't know why he said my name.

Yeah, Jensen called me. I told him not this time, maybe next time.

Yeah, yeah. It was, it was, I knew it was going to happen. I was, I was unbothered. Didn't matter to me, you know?

Yeah. No, seriously though.

Um, it was sick things, man. Yeah, big things. I'm excited to hear what you guys are going to go forward in the future.

Yeah, we'll announce the fundraising tokens and then you'll, uh, you'll, you'll, you'll, you'll see.

Awesome. We encourage every new startup to do this, announce their fundraising Claude 4.6 fast mode tokens. Thank you so much for joining us.

Amen.