Own or Be Owned: Why Every Company Needs Its Own AI Model (Yash Patil, Co-Founder & CEO of Applied Compute)

there is truly some atrophying happening by relying too much on AI tools. The one thing I'm worried about people losing is this ability to actually deeply think about how things should be architected and designed. The best engineers right now are the ones who were like, learned how to code before AI, because at the end of the day, I think humans will be the overlords over all of these AI tools and you will still need to do critical thinking to know what you want to build. Palantir is one of the few companies that signs massive contracts without knowing if they can solve the problem. They'll sign like huge, you know, $100 million deals before they've Started working out the problem, right? Yeah. I think we fit into that category where there's open research questions and people who like to work on things that are uncertain, just naturally attracted to companies like AC. I know there's a lot of folks who are like, oh, 50% of jobs are going to go away in the next 2 years. Personally, I think that is not going to happen. I think just the diffusion of AI into the real world takes time.

Demand for intelligence is outrunning the chips that supply it. Meanwhile, tokens are being sold at subsidized prices that won't last. Demand keeps climbing, and companies are discovering that using a frontier model for every task is like cooking with a blowtorch— spectacular and the wrong tool for most jobs. Yash Patil lives at the center of that squeeze. The 23-year-old ex-OpenAI researcher runs Applied Compute, a $1.3 billion company that trains custom models on a business's own data— smaller, cheaper, and built for the work that, that company actually does. One year in, and Yasha's startup is already serving a mix of tech startups and established corporations, helping them enter the AI era all while owning their own intelligence. In this episode, we discuss why open-weight models are quietly taking over real workloads, why Yosh believes AI's transformation of the economy will take decades rather than years, and what it was like inside OpenAI the weekend the board fired CEO Sam Altman. I'm Mario, and this is The Generalist. The best founders aren't spending their time on expense reports. They're busy building. Brex is the agentic finance platform that makes that possible. High-limit corporate cards, banking, and AI that handles the back office automatically so your team never has to. Expenses get captured, books get closed, and spend stays in policy without anyone chasing it down. Your team gets their time back to focus on what actually moves the company forward. Vercel, OpenAI, Anthropic, Granola, and Deepgram already run on Brex. 1 in 3 startups in the US does too. It's time to get Brex. Go to brex.com/solutions/startups. Every revolution in AI creates one question that never changes: can you trust the output? AI for work is incredible, but without trust, it's just leading to faster mistakes. The challenge isn't building an AI that can answer questions, it's making sure those answers are right. That's where Guru comes in. It's the AI source of truth that connects everything your company knows. So every insight, every answer, every recommendation is grounded in verified knowledge, not outdated information or hallucinations. When your teams and your AIs share one trusted foundation, everything moves faster with fewer redos, fewer blind spots, and more confidence in every decision. Because in the age of AI, Truth isn't just power, it's protection. See what Guru is doing for thousands of companies like Spotify, DHL, and Stripe at getguru.com. That's getguru.com. Yash, lovely to have you here. And we've been chatting already sort of before we've sort of kicked this off. And I think it makes sense to maybe start with some of the things we were chatting about if that sounds good to you.

Of course, yeah.

Yesterday, Fable 5 came out. Big day in the world of AI and has implications for all of us, but including on the sort of the business you're running. I know you've got a few thoughts on it. What came to mind for you when you sort of saw it?

I think first of all, the model is amazing. Like a ton of our team members were using it. It's like clearly feels like a step function. there have been moments in time where we've seen better and better models, and this is definitely like a point in time that should be noted. I think though, the thing that you're referring to is kind of when we were looking at the system card, and this was kind of blowing up on Twitter and whatnot.

Yes.

There were some stipulations about how you could use the model, particularly some of the guardrails where the model might not answer in a super intelligent manner, or it might sort of like, answer around particular types of questions. And I think in the past, it's been focused on things that are very reasonable, like safety concerns and things like that. Things that kind of our consensus view, models helping with this kind of stuff is not great. But yeah, in this case, it was queries related to AI model development, which really sparked this question of like, Hey, how much control do the frontier model providers have in how you use these models? And if they do end up holding out capabilities for particular types of work, what does that mean for the downstream users of these models? So the natural conclusion and what the core thesis of our company is, is own or be owned. Having ownership over your own model and your own intelligence is actually really existential for a lot of people. And if you don't start investing in owning your own models, you might kind of be in a tough position when these things do change underneath you. It's kind of like a model-less company is sitting on shifting sand.

Yeah. And I think the consensus things that people understand that Anthropic might not want to help you with is like biochemical weapons and such. But folks that are relying on these models to do more AI research, discovering that that is sort of maybe in some places subtly being degraded is difficult to adapt to.

Totally. And like, you know, there are stated reasons for why they're not doing this, particularly around like, you know, safety in terms of model development where it's unclear whether having everyone have access to these tools is beneficial for humanity. But the big thing is, who's going to make that call? And the people who make the models are going to make that call. And then the not-so-charitable view of this is Anthropic is doing very well and making some extremely powerful models. And this kind of prevents other people from being in that race as well, if they want to use Anthropic models to build up their model development stack. So it's a little anti-competitive. Yeah.

Yeah. My view would be that I think they really believe it. Like, I think they sincerely worry about this stuff. Like, you know, everything they, they put out into the world, uh, is fairly consistent around that. Do you, I don't know, do you find it to be more of an anti-competitive behavior?

No, I, I, I, I don't, I actually don't think it's anti-competitive. I think it does come from a really good place, which is like, I have friends at Anthropic and I really do respect the leadership. They handled the whole Project Glasswing and deployment of Mythos really well. They do get some flack for it, but I think if you do have a powerful new technology, it makes a lot of sense for you to take things slowly and roll it out to preferred partners first. Also, with this new Fable model, there's some stuff around data retention. Which makes sense. Like, you cannot cover the full surface area of what these models can do just within the walls of Anthropic. So you actually need to see how people use it. I think the important point is, and these things can be both true at the same time, I think, you know, Anthropic is acting in really good faith, but also at the same time, it does raise the concerns of like, hey, if you don't have control of your model, if Anthropic ever does want to hold out a capability for competitive reasons, Yeah. What happens to the users of those models? Because I think Anthropic is an amazing business. They're obviously doing incredibly well and they're moving into the application layer and things like that. So there are products that rely on Anthropic models that are going to be competitive with whatever Anthropic puts out into the application layer. So I guess the question is what happens when, You're relying on those and they want to push their products higher up the stack.

Yeah, there was always the phrase, I think it was around Apple when, you know, they would release something that sort of immediately threatened something within their developer ecosystem, it's Sherlocked, right? If you would say that it's like Anthropic, every time these, one of these models comes out, it's like immediately someone is, is depositioned potentially. And in these different ways to come back to the idea of owning one's intelligence, Maybe we should talk about it in the context of your company, Applied Compute. Like, what does that mean? Why is that so important? And why is that the challenge that you're spending your time on?

Yeah, so what we do at Applied Compute, maybe I'll give a little bit of an overview, is we help companies own their intelligence by training custom models on their data, being able to serve these in production at scale, and then continuing to improve them the more they're used. In practice, what that looks like is we benefit from the open source community, basically taking open weight models that are extremely powerful, actually. I think if you sort of track the development of open source models compared to closed source models, they've kept up and a lot could argue that they've sort of been closing the gap from when the first open weight models started to come out. But yeah, we basically are able to harness these open weight models, combine it with customer data to create task-specific or specialized models. So the idea really comes from the fact that we don't think all AI workloads are homogenous, right? There are places where you want to use the right model for the right task, and you want to optimize for cost, latency, performance, modality, deployment flexibility, that kind of stuff. So our vision is that AI is actually going to be very decentralized in the sense that many companies will own their own models rather than there being a sort of single model that does everything.

Yes. And so to put it just in probably a too dumb analogy for folks, it would be sort of a Goldman Sachs of the world has all of this useful data. I think you've even used maybe Goldman as an example before. That's top of mind. Has all this useful data context. And, you know, just unleashing a general model into that loses a lot of what, you know, the value of this place is. And you also don't want to necessarily give that data away.

Totally. Yeah. I mean, the way these companies operate on the inside is like a lot of what makes them special and differentiated. So, you know, I had Brendan, a good friend of mine and CEO of Merkur. He was the one talking about the Goldman stuff. He was over at our office last week talking about what they're seeing in enterprises. And I think the manifestation of what differentiates a company when it comes to AI is coming up in this talk around evals. So companies have their own definition of what good looks like, how they do certain tasks and procedures inside of their company, and they're codifying it within evals. And there's no way they would ever sort of give those evals away to the model providers because that's a lot of their secret sauce. There's some things that they might want to share to sort of improve the capabilities of the models, but anything that's sort of proprietary and unique to them, that's what they're going to want to hold close.

So many of these topics are, I feel like, so live right now, like the question of the value of routing and cost and open source.

Tokenpocalypse. Yes. Yeah.

Um, so I want to come back to it, but maybe to sort of set the stage a little bit, I'd love to hear about your story a bit. I was seeing that when you announced Applied Compute for the first time, uh, there was a tweet from like someone you went to middle school with, which was very sweet. And it described you in such an interesting way that it made me very curious. Describes you as the middle schooler hanging out with high schoolers and someone who had a very unusual sort of unconventional mind.

Yeah.

Yeah. What were you like as a kid? Were you just building things from the very beginning?

Yeah. So that's actually really funny. I know exactly what you're talking about. It's a friend of mine. Her name's Sarah. We went to high school together, but she was a few years above me. So, you know, on a little tangent, there was this club called Science Olympiad. And we had like, there was like this middle school where a lot of people went to high school, like this specific high school afterwards. She was there in high school, I was there in middle school, and those Science Olympiad teams often hung out together. So that's how I got to know her before I was there. But yeah, in terms of childhood, I can tell you a little about my family. So, small family of 4. Mom is a pediatrician, runs her own sort of medical practice. Oh, wow. Uh, my dad's like an electrical engineer, works on chips and things like that. Very relevant now. Um, and I have an older brother, uh, who, who, uh, is in tech and, uh, is like a software engineer, now works at a really amazing company called Chai, Chai Discovery, plug for Chai.

Yeah, I love it.

Uh, Chai Discovery. Um, but growing up, yeah, like my brother was actually the person who got me into, into coding. So I feel like older brothers always like get way less credit than they deserve. Like, they really paved the— or, you know, older siblings in general paved the way for the younger kids. And yeah, he was the first to sort of start fiddling around with, like, building apps and games and things like that. And one of the things he did sort of when he joined high school is he, like, built this grade-checking app. So the grade-checking system in our school district was terrible. Everyone hated it. So he just built this app and then got a bunch of his friends to use it. Eventually the whole school was using it. Eventually a bunch of schools in the whole school district were using it. And I just thought it was so cool that he, as a single person, was able to build this cool thing. And now everybody I know is using it. So that really got me into this mode of like, oh, wow, I want to do that too. Basically classic copy the copy the older sibling. So my dad actually encouraged me a lot to build these kind of, like, little fun, unique apps or platforms and stuff. But he doesn't know how to code, so he would give me the ideas and then he'd say, go build it. I would go and build it and show him, and he'd be like, oh, you should do this or that or something. So, like, you know, for example, so my dad grew up in India, like a village, and it was very communal type living. And the way it would work is each family would, like, cook a bunch of food of the thing that they were really good at cooking. And then they would go and trade food with other families and things like that.

How sweet.

Yeah. So he was like, oh, you should just— your mom makes really amazing burritos. And then we know this neighbor makes really good barbecue. So he was like, you should go and build something that helps us trade food or whatnot. Built it. It was really fun. No one ever used it. But it was—

Weird in America.

Exactly. But it was a lot of fun to just kind of like iterate on this thing and see something come to life. And that project actually got me my first internship.

Oh, wow.

Yeah, my neighbor, he's in tech, and he was working at this small fintech startup that was like 4 people. And my dad was like, in classic dad fashion, was like in the yard talking to him about, oh, you know, Yash is working on this thing, yada, yada, yada. And he was like, oh, do you wanna come and work at our startup for the summer? And I was— I'd never done anything like that. So I was like, oh, that's super cool. This was like kind of freshman year of high school. So yeah, he took me to their office, which was just an apartment building, like one unit there. And I thought it was super cool to see like all of these just normal people, like, Working, like, it's such a small group of people working on this, this sort of like hackathon project. And yes, you know, it was, it was very different than when I visited my dad at work and he's, he's in the cubicles and like, yeah, like 100 people and things like that. Yes, it was like, I was like, cool, this is like a bunch of people who are doing exactly what I wanted to do in my free time with my dad.

Yes,

but

as a job.

Doing it as a job. And so I was like, this is super cool. So Uh, interned there and a couple other places. That's how I kind of got into startups and stuff. But, um, yeah, it was kind of cool.

Amazing. And then you, you go to Stanford and I think I've seen you say somewhere that, um, you know, you're a very good student in middle school and high school. And then when you came to college, it was sort of like, uh, you know, I've arrived and I'm going to basically watch the lectures and just build stuff.

No, no, exactly. So, I mean, in, in middle school and high school, Grades were super important. I think it's like a common thing in a lot of immigrant families as well. But yeah, when I went to college, it was kind of like, give me an inch and I'll take it. So I was like, oh, like, you know, classes are online? That's cool. I'll just like watch lectures online. I spent most of my time just like hacking on like basically manifestations of what I was working on in like childhood, like middle school and high school. Yeah. But for like the university setting. So nothing, I wanna be very clear, none of this was impressive or, like, interesting or anything like that. I was, like, working on, like, every single generic college campus coding project you can think of. Like, social calendar for, like, me and all my friends.

You have to do these things, right? You get these reps.

Exactly. Or, like, a dining hall voting app or something like that. 'Cause, you know, some food is better than others and things like that. But yeah, just through that, like, found other people who were kind of doing the same thing. I think Stanford's a really cool place because there's these sort of natural communities that form and there were little informal clubs of like, oh, let's get together and hack on a project or something like that. And then sort of the main organization I was affiliated with at Stanford was TreeHacks, which was the school's hackathon that we throw for everyone across the nation and even some folks internationally. Yeah, spend most of my time just like kind of working on projects and stuff rather than attending class.

You'll have to correct me if I'm wrong, but you're a Thiel Fellow, but you also seem to have gotten a degree. How can this happen?

So I did not get a degree, actually. Yeah, yeah. So I dropped out of Stanford during my sophomore year, about halfway through my sophomore year. So yeah, was basically there for a year and a half, dropped out to join OpenAI though. And then after OpenAI started, sort of applied.

And OpenAI, I think the way you've talked about it before is like you were just so mesmerized by this technology. Like, can you, yeah, tell me about that?

I mean, I look super fondly upon OpenAI because, yeah, obviously the technology was amazing, but the people were like, Even more amazing. So I actually didn't really take any AI classes when I was at Stanford. So joining OpenAI is how I learned a lot of this stuff. And there were incredible people on the post-training side that kind of took me under their wing and taught me a lot of stuff. So Luke Metz, Baratzov, a bunch of folks on the post-training side were like, they were my managers. And I was just kind of looked up to them. They're legends in the language modeling space. They made the first versions of ChatGPT with John Schulman. It was actually really funny. So Barrett, who used to be my skip, told me on his last day when he was leaving OpenAI, he was like, Yash, by the way, I don't know if you know, but when you were joining, you were originally supposed to be on the ChatGPT engineering team. And we just really needed more people on post-training infrastructure, which is on the research side. So it was like last minute we pulled you in. And I didn't know, because you don't know your team before you join. And it kind of, that's basically the reason why I got into all this AI and research stuff is completely by luck.

Yeah. It set a few things in motion.

Yeah. It set a ton of things in motion. I feel super grateful for the people that were there. And then the fact that They were willing to teach me a bunch of stuff and sort of take me under their wing. And yeah, I mean, this guy, Iain Osborne, who's now at DeepMind, threw my 21st birthday party. It was a lot of fun. I was lucky to be surrounded by some really, really awesome people.

Am I right in recalling that you got the OpenAI job by cold emailing Sam?

Yeah. So the backstory there is rewind to freshman year. We had been hacking on some projects and stuff, and my friend and I both had internships lined up for the summer, but we were like, oh, let's just keep working on cool things and keep hacking on stuff. So Stanford is a very unique place because I think there are a lot of people who encourage folks to do this, to go and just build stuff. So there were a few venture firms that were like, hey, here, take an angel check or something like that. Yeah. and go and spend the summer building something. And then if you end up not wanting to continue, it's totally fine or whatever. But growing up, I knew that nothing comes free. So prior to this, I had been invited to Sam's place for a dinner. He puts on some dinners sometimes for people who are building stuff on campus, whatnot. There were, I think, a couple dozen folks who went. And Sam is someone I deeply, deeply admire. I think he's amazing. And so I had watched all his YC videos and read all his blogs and stuff like that. Couldn't work up courage to actually talk to him at that dinner, but a couple weeks later when we were like, oh, should we do our internships or not? Shot him an email and was like, hey, there's some people who are offering to sponsor us for the summer. We don't really know who who they are. We're not even sure if we feel comfortable. Like, what do you think we should do? Do you know these people? And he was like, you should totally do it. If you're— this is back when Sam was kind of talking about like, oh, people should spend more time building stuff and take time off of school to like try and explore things. I see, yeah. He was like, if you want, I can basically give you a small stipend, um, to just like pay for rent and food to go and work on it, and I'll sponsor you for the summer, which is like deeply generous of him. And we were like, wow, this is cool. And so yeah, so we just did that for the summer, worked pretty closely with someone who was running sort of his family office at the time, who became a friend, and then ended up shutting that down though. It was fun. It was like an e-commerce company, but ended up shutting it down and coming back to school and returning the the money and things like that. But yeah, that's, that's how I got to know him. Um, and then when ChatGPT came out, I was like, wow, this is the coolest thing I've ever seen. I have to work on it. So then I was like, oh, hey Sam, I don't know if you remember me, but like, you know, I think ChatGPT is super cool. I'd love to work on it. Like, are there any ways I can apply for a job? And then he was like, oh yeah, like we have this residency program. You should apply for it. So he introduced me to the residency folks. Mm-hmm. Talked to them. They were like, oh, you have to drop out. And I was like, I'm fine with that. I can take time off and stuff. And they were like, no, you really have to drop out. Like, you have to like commit to never coming back.

Burn the boats.

Yeah. And I was like, I don't know if I could fully convince my parents of that. And then, so I was like, Sam, like, you know, should I do it? And he's like, let me chat with them. The residency is like a 6-month thing. So, you'll have the option if you want. Then I interviewed, uh, was very lucky to get the residency position and then joined like basically like a week after.

They needed you to drop out like almost as a proof of conviction somehow?

I think at the time it was just, it was actually just a much smaller company, right? So like, I think they're, they're, it's not the, you know, massive company that it is today. This was actually something that I don't think they had done before. Like they hadn't had people do the residency that had like dropped out of college just yet. But after I did it, I think there were a couple other folks who became more mainstream. But yeah, I guess I think they just wanted commitment and stuff. But I already kind of knew at the time, I was like, I would be so lucky to be able to work here. I totally would, if it was up to me. It was not up to me. My parents had to have a say, right? So.

Clearly the right choice.

Yeah.

You ended up living through one of the most interesting periods of tech technological history ever, and also one of the most dramatic, like, weeks of technological history ever. I'm sure there's lots of that you don't want to talk about, but you were there when Sam was ousted, came back. What was that experience like for you as someone having what was, I think, their first proper job? Like, that's a very dramatic thing to go through.

It really sucked. So I remember that week pretty, pretty well. And I don't, I don't want to be overdramatic or anything like that. Things happen and stuff. But it was kind of this feeling of like, oh my gosh, things are going so well. We're making so much progress. Everything's happening. Why are we halting the progress? Let's just go. Things are going so well. And so I just remember there were a bunch of— basically that whole week was actually like, we couldn't go into the office. All our laptops were shut down for security reasons and whatnot. Oh, wow. It was like every day there'd be someone who'd be who was like, "Oh, I'm like hosting at my place. Just come hang out." Because we no one had anything to do at work. So so yeah, I just remember like everyone being down on the fact that it wasn't even about like the sit like you know everyone wanted Sam to come back and no one more than me because I think he's truly awesome. But everyone was kind of sad about the mission just being on pause. They were like, "This is." this is weird, like what we were doing so well. So, and then it was a roller coaster too because, you know, Sam was talking with the board and things like that. And, you know, we all thought he was coming back and stuff. And then there is an announcement that, you know, he wasn't and there's someone else stepping in. But eventually, right, I think, you know, we were super excited that Sam came back as CEO. And it's actually funny, I remember So after that was announced, everyone was gonna meet back up at the office. And I lived, so I lived across the street from the office in this like the building, the Madelon. And so I like came into the office and Sam was actually coming down the stairs. So gave him, he gave me a big hug and I'll definitely remember that for a while though. That was like probably one of the coolest, coolest days was just like, it's back on track. Like, let's go, right? So.

It sounds like, you know, there's obviously been, I'm thinking of all the stories that have been out over this year, the books. Sounds like you still are basically, yeah, a big supporter of Sam and think he's a good executive in that respect.

Oh yeah, I think, I think, yeah, a little side tangent on Sam. I think he, he's an amazing CEO and his superpower is he's able to make impossible things sound extremely possible. Like, you know, if I were to tell you, hey, we're going to spend hundreds of billions of dollars to build the biggest supercomputer in the world and train a model that can do 80% of all knowledge work, you'd think I was crazy. Yeah. But somehow he's able to go up in front of hundreds of the smartest people in the field and be like, guys, this is possible. We can do it. And I think that extends to the whole leadership team as well. Ilya is really, really good at this too. He had his, uh, feel, you know, the Feel the AGI stuff. Like, yeah, I really admire Sam's ability to make impossible sounding things sound possible. And that's really how you get the best people is you work on really hard problems because all the best people want to work on hard problems. No one wants to work, no one who's really, really smart wants to work on things that are, that are easy.

I think that's a, yeah, I mean, that is a skill. I won't make you answer for For Sam's journey. In terms of your time at OpenAI, you mentioned that you ended up spending it on post-training, and this sort of had such an impact on what you're building today. For folks that, I'm sure most of the audience or a lot of the audience will be really intimately acquainted with this, but for folks that maybe aren't, can you explain a little bit about why post-training matters and how it sort of led to this? work you're doing with Applied at the moment?

Totally. Yeah. So kind of a high-level overview of training models in general is there's a couple phases of training. A lot of people talk about the big, expensive, high CapEx phase, which is pre-training, which is let's go and take internet-scale data, use this architecture called the transformer to sort of learn language patterns. And out of that falls intelligence, right? So you get this model that can do next token prediction. And that's kind of like how we got the first sort of GPT models where they just got really good at predicting the next token. And it felt like these models could actually reason and answer questions and things like that. After you do pre-training, you basically have an unaligned model that will just predict the next token until you tell it to stop. To make it useful, that's when you move to things in the post-training realm. So there's a couple of different types of post-training. When we were first training these chat models, where it's like a user sends a message, an assistant sends a message, there's a variety of techniques of supervised fine-tuning, as well as reinforcement learning with human feedback to steer the models to give responses that are sort of coherent in the format and tone that a human would like and whatnot. But you're sort of just basically doing behavior cloning on the supervised fine-tuning side. So you're saying, here's a bunch of examples for good answers. If someone inputs this prompt, these are examples of what good completions look like. Yes. And then you just train the model to reduce the loss on those target tokens. Now, what people have been talking about when they say post-training today is something quite different than supervised fine-tuning or reinforcement learning with human feedback. It's this idea of RLVR. So reinforcement learning with verifiable rewards. It turns out if you take a really small, like a relatively small amount of high-quality data as compared to like when you were doing pre-training or SFT or something like that, And you actually have a model try to attempt the same problem a bunch of different times, and you have a verifiable way of checking if the model gets the answer right, you can use this new reinforcement learning algorithm to sort of show the model, hey, these are examples of what good looks like. These are examples of what bad looks like.

Yes.

And sort of nudge the weights to do more of the good reasoning versus bad reasoning. Thing that people talk about when they talk about chain of thought. So all these tokens before the model outputs an answer is actually entirely an emergent property. Turns out, like, the models learn to steer themselves, learn to think, and that's how they get more accurate and get, you know, basically like hill climb any sort of task that you give it. So what the first domain that this started with was math. Part of it is because, so first of all, it's got a lot of really good properties, which it's easily verifiable. You don't need that many complex tools or things like that to be able to do math. And the second thing is just everyone on the research team at OpenAI was competitive math people or things like that. So it was a domain that they deeply understood and also were very excited about. And there's also a lot of other physical philosophical reasons about why math is kind of the root of a lot of other things. So basically the first experiments were actually hill climbing on a variety of different math problems ranging from basic arithmetic to harder proofs, algebra, calculus proofs, whatnot. And turns out that the model is able to learn on the easier problems and internalize that and be able to unlock harder and harder problems. So that's this Hill climbing that you've seen. And there was kind of a couple of scaling laws that came out. So obviously you had the pre-training scaling laws. Now you have this post-training scaling law, and now you have this test-time compute scaling law, which is like, if you actually allow the models to reason for longer when you're using them in production, they give you better answers. So that's this, when you pick a reasoning level in ChatGPT or something like that, that's it just reasoning for longer, expending more test-time compute before. Yeah. Giving final answer.

This episode is brought to you by Persona, the B2B identity platform helping businesses verify users, fight fraud, and build trust. Fraudsters are already using AI to spoof faces, voices, and documents, so your defenses need to adapt just as fast. Persona helps secure some of the internet's largest and most trusted platforms with identity verification. If you're building a product where trust matters, identity should be a priority. You've probably already experienced Persona without realizing it— verifying your LinkedIn profile, signing up for Etsy, or renting a scooter with Lime. Trusted by leading companies like Square, Brex, and Twilio, Persona gives you the building blocks to create identity flows that adapt to your customers, risk tolerance, and locales you operate in, whether you're verifying age, onboarding businesses, or automating KYC. It's fully configurable, so you can launch in days, not quarters. Want to see for yourself? Generalist listeners get a free year of the starter plan. Head to withpersona.com/generalist and check it out. You're doing this work at OpenAI. What was the moment that you realized there's a need for the sort of service that Applied Compute offers, where you are sort of going into these companies and doing this more hands-on work?

So I think Microsoft AI, when they put out their thinking models recently, the title of their paper is like the hill climbing machine or something like that. I think the way of thinking about RL is it really is that. It's like if you define the hill to climb well, the algorithm works. So the way we were training these models was actually just like on the post-training side was, hey, let's go and make a bunch of high-quality datasets that target the capabilities that we want the models to get really good at. Basically, we'd start with an eval and be like, okay, if this is what the eval looks like, let's just go make training data that looks like the eval.

Yes.

And that was the way that we were improving these models. So the natural conclusion was kind of like, hey, everyone is going to have their own evals and the things that they want these models to be really good at. And evals extend beyond just the intelligence of the model. It's like, oh, it needs to be this intelligent in this cost and latency regime or this modality or something like that. So if everyone's going to have their own evals, everyone should also be able to train their own models that do well on those evals, right? So our sort of natural conclusion was like, hey, maybe since we have this hill-climbing machine, the best way to sort of manifest it in the world, obviously the frontier models are amazing, Um, you know, these will be like workhorse models for a lot of things.

Yes.

But if people want to be very specific about the task that they're trying to optimize for, they should have the ability to go and push train their own models on their own data.

Yeah. Eval's are, you know, the, the scorecard for all of these models. And that's so general, but it's the

new product. It's the new PRD.

Yeah.

Yeah, exactly.

And, and the reality is that every organization has a different scorecard for what they consider valuable. And that's you know, sort of what you're saying is like, you know, if you have a different set of things that you're optimizing for, it starts to make sense that you have a machine that is made to hit those optimizations.

Exactly. And if we go back to the Goldman example, right? Like if a lot of those evals are going to stay private, people are going to want to own their intelligence, own the post-training capability to go and create those models that are good at those evals. Right. Yeah. So I think that was the primary motivator. And then I think the main sort of driver for why people are thinking about this right now is cost. The tokenpocalypse has really shown, hey, you need to use the right model for the right task, even beyond just optimizing for evals, right?

Yeah, that's— I'm curious how you think about it today. Clearly, that's top of mind. I think in the past, you've talked about how general models sort of set the floor and specialized models can sort of raise the ceiling in some sense.

Yeah.

In some ways it feels like, you know, the distance between those two might be getting smaller, but the cost at which you are accessing these different offerings is sort of like, has it become more of a cost story in your head?

Cost has become like a, the main driver, I think, right now. But I think in principle, the argument still stands, which like, take Kirkland and Ellis, right? They caused waves in the Twitter community, and I'm sure broadly as well. about how they're going to spend half a billion dollars to— I forgot what the exact quote was, but essentially the idea was to create AI tools that their competitors can't also use. So I think the idea is, and this extends beyond just the model, to be clear, the model is part of the moat, but also training that model inside of your harness with your context, that's actually how you're going to get differentiation. Yes. I think a lot of people are thinking about, hey, if I'm using the same stuff as my competitors, how am I different? And that's also another driver for why they should train their own models on their own data.

And I think it's been just almost a little over a year, right? Yeah,

yeah.

And you've gone from starting the company to a last valuation of $1.3 billion.

Yeah.

Been a good year.

Been a good year, yeah.

Can you tell me a little bit of how you've started to work with folks and what that process looks like with these companies?

Totally. Yeah. So we work with a variety of companies across AI natives, digital natives, and large enterprises and hyperscalers. So basically, if you look at how money is spent on these models, there's some amount spent on training, and it's a non-trivial amount, but most of the money and most of the usage is on inference. Yes. So when we were thinking about what the future of open models looks like, or models that people are going to inference a lot for different things. Our conclusion was kind of, if you are going to train, or if you are going to use open weight models and specialize that model, you're going to do some sort of training on top of it. So the actual wedge into inference is getting really good at training. So what we do with our customers is we We have a post-training platform where they can integrate all of their data, and we have a team of applied researchers that actually deeply embed with the customer, work with their ML and research teams to train these custom models. And we really want to make sure that the economics are in their favor. So we try to have most of the stuff that they pay for come on the inference side. Right. So it's like once we have a really good model, And you guys want to use it, then we can go and host it for you.

Yes. The sort of forward-deployed engineer model is something that, that I think people obviously associate with Palantir as a successful example of it. With Palantir, I think, you know, the way they've made it work is that clearly they found a way to get good leverage on it such that the 50th version of it, of, you know, them taking on a project is very different than the 5th. What's the version of that for you that, you know, stops it from being sort of just pure consulting leverage?

There is always some component of this that is going to be unique per company. That's kind of the whole thesis, right? It's like each company is going to have different data and want a different model, but really it's the infrastructure that is repeatable across all of these companies, right? It's like the scaled infrastructure for training these models, the scaled infrastructure for serving them, and then the sort of techniques for capturing production usage and turning that into more data that we can go and continually train. So our equivalent of Foundry, which is Palantir's platform that they use to give to all their FDEs to go and move faster.

Yes.

We have our own post-training platform where all of our applied researchers and researchers from the companies we work with can both collaborate to train different models. And really, I think the way of thinking about it is it's It's the infrastructure to make training performant and easy.

Does it end up becoming more of an ongoing engagement than people might realize from the outside? I could imagine that one version of it would, people would think, oh, you go in, you're sort of doing an AI transformation and then you're out the door. But I would imagine the demand for improving this, deepening it is like leads to quite long engagements.

Totally. Yeah. I mean, so I think there's always a desire to continually improve these things, which is why, you know, when we deploy a model in production, a lot of our product is built around, hey, how do we take the usage of this model and turn it into more data that we can continue to train on or build context around the model so that, you know, it's learning actually, the more you use it. So yeah, I think the idea is that the engagements don't end. Obviously, like, the way we generate revenue is by serving the models, and we need to make sure that those models are always performant.

And maybe even to get even more sort of tangible, like, can you maybe talk us through an example or two of like what taking this more specialized contextual approach unlocks for a customer, for instance?

Yeah, so I think one of the case studies we have on our website is this engagement we did with DoorDash, which is, they have more than 100,000 merchants that come to their platform every year, and they want to create a new DoorDash storefront, right? So when they do this, they come with a bunch of unstructured information about their business, what food they serve, things like that. One of the things is a picture of their menu. So it turns out this was actually an example of where we were able to exceed the frontier in terms of quality, is we trained a small specialized model that was able to take pictures of menus and turn them into high-fidelity DoorDash storefronts. Basically, the process was create an eval for what good looks like.

Yes.

Create the training data for for us to go and like do RL training on top of, train the model, and then deploy that and host that.

The trend feels like, you know, if applied compute continues to grow, and this is part of sort of a broader movement, is like sort of trending towards more of these small models where you do sort of say, here's a set of tasks that like are maybe a bit more narrow, a bit more deterministic, that like we can actually very inexpensively do this, very effectively. Is that roughly how you see things going?

Yeah. And so I think the motto is better, cheaper, faster. What people are starting to realize is that these open weight models provide a ton of flexibility around ownership and they're actually good. Like you can actually go and use them to perform optimally on your task. So I think you're going to see just a lot of people for reasons related to cost. I think cost is the big driver right now, but there's just so many other downstream benefits you get by owning your own model.

When you sort of move towards the world of, you know, a company having multiple of its own models for different tasks, does that create like some sort of maintenance debt for them that they have to sort of stay on top of? And yeah, I mean, there's probably still cases where that's very much worth it for them, but is that something you've noticed come up?

Yeah. I mean, so this is part of the continuous improvement part is there's obviously model drift and the distribution of tasks might be changing slightly, but over a long period of time, that really, really matters. Our view is just training and inference loops, they're like this right now, but they're starting to move closer and closer together. Right now the form is people are just getting a lot faster at making new data and training the model again. But eventually these things are going to be quite intertwined. I think there are some cool examples of this that are kind of like maybe the first sort of beginnings of it. Like if you saw Cursor's Composer's real-time RL work.

Yes.

I mean, that was them doing online training by taking a bunch of user trajectories, extracting the implicit reward, taking a training step, and then deploying a new model in production. I think like that is just kind of an indication of where things are going to go is like you're actually going to be doing more and more continuous training because as your model is being used, it's kind of like an exploration technique, right? Yes.

And so what you're describing there is almost like a continuous learning model where it's like every time I happen to do this task, it's actually training this model to be better the next time around.

Totally. And the name of the game is learning from sparse rewards, right? So if you think about the different phases of training, pre-training is super data inefficient, right? You need to learn the whole— or you need to train on the whole internet in order to learn the language. Then we had this SFT stuff, which was kind of behavior cloning. So you say— I don't

know what

you're talking about. Yeah. So SFT is described as behavior cloning because you're giving it examples of what the correct output should look like. And so you give it a bunch of examples, and then for a new task or a new prompt, the model is sort of interpolating, like, oh, I should say these things. Okay. And then we kind of moved to RL, which is a lot more data efficient, which is I'm going to take one data point, one math problem, generate 1,000 trajectories, all with different thinking tokens or whatnot, all with different rewards, And then I'm going to do backprop on those. And there the model will kind of learn what good reasoning looks like and bad reasoning looks like. The ultimate thing is being able to do super, super sparse learning, which is like you go and roll out a trajectory, you have some indication of whether it was good or bad or something, and you're able to directly learn from that. That is difficult, and that is the problem to solve in continual learning.

On the open weights models, you were mentioning earlier that you think maybe The gap between open weight and closed models has sort of narrowed over the past year. In some ways, if I look at like the hardest benchmarks, I almost feel like it's widened, but actually what we're learning is that it does just beyond a certain threshold, it almost doesn't matter.

Yeah.

Like, I don't know. Do you think that's wrong?

Yeah. Yeah. So, so I think what we were talking about before is frontier models are amazing and you should still use them for your hardest tasks. But if you have like $100 of spend, maybe $20 of those dollars go to the frontier models, and the $80 can be actually $20 that go to cheaper, better, cheaper, faster models that you train on this task, right? So I think like the frontier models are obviously very, very, very good, but you don't need to use them for everything. There's all these memes on Twitter, right? Like where people are using some big sword to cut like a strawberry.

Yeah, or a blowtorch for, you know.

Exactly. So I think like, you know, there's a lot of truth to that. And that's what, you know, you were talking about model routing and things like that. I think there's going to be some pretty interesting innovations there. We're working on some of that as well.

The other thing I think about with open weight models is like China has really dominated that so far.

Yeah.

Um, you know, it feels like America is, is clearly leading with some of the frontier models, but, uh, yeah, it's like, how do you think about the fact that an implication of this could be that a lot of American companies become like actually quite dependent on Chinese models that the rules could get changed up on them. Those could go closed at some point. Like, is there a robust enough open model, open weight system right now?

So, so I think that, you know, you're right. It, it all boils down to incentives. So I'm extremely excited that companies like NVIDIA are investing a ton in open source models. And NVIDIA has a real reason to do it, right? Like the dream for NVIDIA Is that every company has their own model and is running it on NVIDIA chips. So I think that there are real, there are companies like NVIDIA, NVIDIA being the big one, that have true incentives to keep models open. And I think there is enough of investment and a lot of smart people working on open models that I think it's going to be a pretty safe and big bet over

One of the things that I think is really interesting about Applied Compute, sort of outside of the technology piece, is that you seem to have managed to attract a huge amount of ex-founders, which is always a very bullish sign, I think, for an investor to see that happen. How have you managed to do that? It's something like 2/3, right?

Yeah, so it's almost part of the business model is Because we're doing this sort of forward-deployed approach where we're embedding deeply with customers, it requires people who are extremely high agency and ambitious and move fast. So I think it just naturally, the kind of work just naturally attracts these types of folks. And then, yeah, I think in our culture, we sort of decided, hey, we want to hire ex-founders who fit this archetype, and they just themselves attract other like-minded folks. So yeah, I think it hasn't been anything really special. We're not trying recruiting from some very specific pipeline or something like that, but it's just this kind of natural snowball effect where the type of work that we do plus the initial team that we got together just attracts more people of the same— Mentality.

At a higher level, like, what are the things that you think are maybe particularly distinctive about Applied Compute's culture? Like, what do you care about that, you know, other startups you know or other places you've worked maybe wouldn't care about as much?

So there's a lot of this like 996 stuff.

Yes.

We're not a shop like that. Like, we, and I deeply respect people who who choose to operate like that. I think the type of culture that we want to embody is people are just very excited about their work. And we have no requirements on how long people work or things like that, but it's a very bullish sign when people are so excited about what they're doing that they want to come in on weekends and sort of spend a bunch of time in the office. So yeah, I think the things that we prioritize or what we look for when people are joining the team is like being deeply mission aligned. So, you know, we want people who, who want to come and work on hard problems, but also really believe that, you know, something like this should exist in the world and that this is actually very important for basically every company out there. And then take a lot of pride in working on hard problems that don't have, like the thing that's very unique about our company, right? And It's actually somewhat related to a Palantir type. I remember having this unique moment where I was like, Palantir is one of the few companies that signs massive contracts without knowing if they can solve the problem.

Wow, how interesting. That's so true.

They'll sign huge $100 million deals where this is before they've started working out the problem. So I think there's a unique set of companies that that do that. And obviously, like, I think we fit into that category where there's open research questions about like what we are trying to go and get really good at and solve. And people who like to work on things that are uncertain and there's a lot of greenfield research to be done are just naturally attracted to companies like AC.

When you talk about, you know, wanting people who want to do the hard thing, like what's been the hardest, thorniest technical piece of this so far for you?

Yeah. So I think one of the really cool places of work in the company right now is basically building the best ML infrastructure for training OpenWIT's models. So we're obviously in this compute crunch. The more performantly you can train models and the more accurate they can be, that's true differentiation, right? There's actual work. It's not like RL is completely figured out and it's kind of like a commodity thing. There's real research happening at the ML infra layer. So we've been hiring a bunch of these really, really smart researchers that are actually able to go deep into the stack, work very close to the hardware to actually make training more performant. And that's one of the ways we're differentiating amongst other things that

are out

there.

Maybe part of this is too secret sauce, but can you talk a little bit about what that training stack is, what that looks like?

Yeah. So basically, we procure our own GPUs, and then when you're actually going and doing training, the way training works is you're alternating between doing inference. So you have some checkpoint, some weight checkpoint that's like policy step 0 or something. You go and try a bunch of attempts at a problem, and that problem is actually in some sort of replayable environment. So say we're training a coding model, You're spinning up a sandbox that has a codebase on it and potentially even more complex than that. You have services that are running that the model may need to interact with, yada, yada, yada. All needs to be replayable though, right? Because you're repeating this like 1,000 times.

Yes.

You go and do the inference on that, get a bunch of model attempts, assign reward to them based off of some verifier, and then use some sort of objective to be able to go and take the next step on the model and get your policy step 1. And so one of the really interesting problems is like, okay, you're doing this sort of alternating of sample, train, sample, train, sample, train, sample, train.

Yes.

It's always slower to do things sequentially, right? So what do you actually want to do is you want to do things asynchronously where you can actually be sampling on your next set of problems while training is happening.

Oh, wow.

And you can actually, there's really cool science questions around this of how far can you push that? How asynchronous can you be? How off-policy can you be? Can I be sampling problems using policy step 1 or something? Or sorry, policy, some earlier policy while I'm training the last policy. Yeah. So this is where a bunch of the ML dynamics come in. How do you make these off-policy corrections where you're trading off performance and speed and throughput? Versus ML dynamics and stuff like that. So these are the kinds of infra problems that a lot of our team is working on. It's like, how do we make this stuff really performant? And it really matters for longer and longer horizon tasks. If you're running your model for an hour, your total training runs can be super, super long if you're doing things synchronously. Yes. Train, inference, train, inference. So you really do want to do things super asynchronous in order to cut down on your total GPU time.

You're sort of optimizing around—

You're layering it.

GPU time, because that's the most constrained resource.

And so this is the other cool thing, right? It's like, so there's that side of the house, but then there's also environment creation, right? Why do people care so much about performance sandboxes and things like that? The reason is because if you're waiting on CPU, workloads. If you're idling your GPUs for CPU workloads, you're probably wasting a lot of money. So there's like a ton of performance engineering on the environment side of the house to make training faster as well.

You know, as you were talking about the sort of imperative for every company to own their own intelligence, it sort of occurred to me, maybe this is a very naive thought, like isn't the sort of logical extension of that, that at some point it'll also trickle down to the human level and all of us will need to have our version of that? Like, does, you know, does AC in 5 years, you know, start to become, oh, you don't need a forward-deployed engineer, but you have a version of it that's like more self-serve? And yeah, you know, each of us has our, our own model. Yeah, that's interesting.

Our own set of models. Yeah, I mean, we've so far, we've been pretty enterprise-facing, but maybe this is, this may be market number 2.

Excellent.

Um, no, I, I think you're right. I think a lot of people have been very excited about this idea of like, personal AI. And I think maybe a bit of a jump to a broader point, but if you think about personal AI, that's something that deeply understands you, how you work, and maybe running all the time. Yeah. So I think maybe a broader point is just like, as these things start to trickle down, the demand for compute and inference is going to balloon like crazy. That's something that's kind of like a broader Maybe I actually don't even know if the market is fully pricing in how much compute that we'll need for all of this stuff. If everyone has their own personal models and that are constantly being trained and inference and things like that, we just need a ton of compute, which

I

don't know,

it seems likely that will happen to me. Maybe that's too— maybe I'm speculating.

No, no, I think people totally want it to happen. The problem is, I mean, I think they're just like, right, there are like physical limits, right? Like to how, you know, how much the supply chain can accommodate. And like, not even like on the chiplet, like, you know, there are ASML machines, like lithography machines that like take a really long time to build. So that's why I think like, you know, we're in for— this kind of ties back to like tokenpocalypse. We're going to be in a compute crunch and inference demand is going to crank like crazy. Yeah. The ability to train Pareto optimal cost performance models is actually going to matter a lot. And I think we're seeing the early beginnings of that, and it's just going to get worse until people really start to invest in this.

What's your sort of, not AI 2027, but your view on the next couple years? What do you think the world is looking like from your perch?

I know there's a lot of folks who are like, oh, 50% of jobs are going to go away in the next 2 years and things like that. I think personally, I think that is not going to happen. And it probably is inspiring a lot more fear that maybe doesn't—

Is maybe misplaced.

Yeah. And I think it's coming from a good place, which is just like, we need to be prepared for this sort of stuff. But kind of being on the front lines, doing this forward-deployed work and whatnot, I think just the diffusion of AI into the real world take time. There's a lot of change management that needs to happen. Companies are large institutions that take time to adopt new things. And we were talking about this before, right? OpenAI is this— it's OpenAI, the research and deployment company. And deployment is so hard that they created a company called DeployCo, the OpenAI DeployCo, right? And same with Anthropic and stuff like that. Yeah, right. I think people know that it is going to take some time to go and deploy this stuff. So I don't think overnight everyone's going to lose their jobs. There will be areas of the economy that I think will experience more change than others just due to the nature of the job. But to me, this is a multi-decade-long rollout and diffusion of AI into the real world.

Wow. What has surprised you most in terms of that change management, maybe the slowness of some of these enterprises? Maybe could you, you couldn't have counted on?

I mean, in, in some ways we knew this would happen, but just seeing it up close, you know, makes a lot of sense is there are just established procedures inside, inside of companies. And one of the difficult things is like, you know, to change those procedures, you often have to touch multiple layers of, of management and, and the actual people who are doing the work. And it is just a difficult coordination problem, right? And I think the other thing that is also obvious, but we've just, Michael Chan on our go-to-market team put out a really good field notes guide to what forward-deployed engineering has been like, is like no one is data ready. Absolutely no one is data ready. Things are obviously always very messy and whatnot. But if data is kind of everything, even just the state of where it is and even the best companies is It's not where it needs to be to actually go and make any sort of deployment extremely smooth right now. So I think there are just these natural sort of roadblocks that will slow things down when you're trying to roll out AI in like big organizations.

Well, I always like to end with a couple sort of thought experiment questions. If you were given the chance to conduct any experiment with no operational constraints and no budget constraints, What's an experiment you'd like to run?

Oh, this is a fun one we talked about in the, in the office, which was if we all reverted to, to cavemen and like lost all modern technology, how long would it take us to reproduce the iPhone? So knowing what we know now, how long would it take us to get all the infrastructure in place? Like no fabs, No, no, no, none of this stuff, man.

That's really

sand. I need to

think about that for like a day at least.

Yeah,

I don't know how— what did you come up with?

Well, so, and, and the— I guess the darker side of this is you have to factor in everything. So like people, you're going to war against each other and things like that. Yeah, if we were just all— lost all modern technology,

100%.

I, I concluded that we just never

You think we would never

have a moon? I think we just, uh, yeah, like, first of all, most of the population— I— this is getting a little weird, but I think most of the population would not just like know how to survive the first like—

Yes, I think that's right.

60 to 90 days. And then, um, you know, we'd have to— there's the ability to go and build all this infrastructure takes a ton of time. You know, if you sort of are thrown into chaos and dystopia, like, people are obviously going to get mad at each other and create factions and fight. And so I think we'd spend most of our time fighting them than actually building Wow, I'm gonna be thinking about that

for a

while.

That's a good one. Okay, sort of last question then, I suppose. If you could assign a book to everyone on Earth and you knew they would read it and they'd understand it, what would you wanna give to people?

I really like this book called The Design of Everyday Things. It's actually one of the first tech books that my brother gave me, and I think it really set like a good framework for how to think about solving problems in, in non-complex ways. And here's like a crazy segue for you, right? Like, I think in today's age when you have AI tools, uh, particularly coding,

yeah,

you can actually, by going faster, you can, you could actually be going a lot slower. So like taking the time to, like, I don't know, the one thing I'm worried about people losing is this ability to actually deeply think about how things should be architected and designed.

Yes.

Because if we rely too much on AI tools, and, you know, I think every company has seen this in some capacity with like this AI vibe coding or whatnot. We're all for AI code. I mean, my main thing that I worked on at OpenAI was Codex, but I think there is truly some atrophying happening by relying too much on AI tools. So back to the basics, back to the roots.

Yeah. I mean, things like judgment, taste, all of that stuff is very easy to, stop actually building when you have these tools in front of you, right?

Totally. I think the best engineers right now are the ones who are like people who learned how to code before AI because they actually deeply understand systems thinking and things like that. And that would be my worry as a new grad engineer or something like that is getting too reliant on AI tools, because at the end of the day, I think humans will be the overlords over all of these AI tools, and you will still need to do, you know, critical thinking to know what you want to build. They might help you build it, but you need to know what you want to build.

Well, I think that's a nice optimistic place to end. Thank you so much. I really enjoyed this.

No, thank you. This was awesome. Yeah. Thanks for having me.

That's it. Thank you for listening to this episode of The Generalist Podcast. Please subscribe on Apple Podcasts, Spotify, or your preferred podcast app. Ratings and reviews help others discover these discussions. So if you enjoyed the conversation, I'd be grateful if you could take a moment to leave one. For all past episodes and more, visit us at thegeneralist.substack.com. See you next time as we continue to explore the future.