Product-Led AI: Adept CEO David Luan on Upleveling Human Work

Hi, welcome to Gray Matter, the podcast from Greylock. Today we're featuring another guest episode of Product Led AI, the new podcast series hosted by Greylock partner Seth Rosenberg, where he talks with leading AI builders who are exploring opportunities in the application layer of AI. This week Seth talks with David Luan who is a CEO and co founder of Adept. The company is developing AI agents for the enterprise. You can subscribe to Product Led AI wherever you get your podcasts and you can sign up for Seth's weekly LinkedIn newsletter to make sure you never miss an episode. You'll find links to all of this and more on the series website, productledaipod.com it's also linked in the show Notes. Now here's Seth with Product Led AI.

Hi, I'm Seth Rosenberg. I'm a partner at Greylock at and the host of Product Led AI a series exploring the opportunities at the application layer of AI. My guest today is David Luan who is the CEO and co founder of Adept. The company is developing multimodal agents designed to work alongside humans in any profession. David has been an early builder, researcher and pioneer in AI. He was among the first few dozen employees at OpenAI where he led all of OpenAI's engineering. He then served as co lead of Google Brain, working on frontier large language models. He co founded Adept in 2022 and the company has stood out for its human centric approach to AI and eventually AGI. David, thanks for joining today. Very excited to dive into the nuances of building agents with Adept.

Thanks.

So maybe to kick it off, obviously you've been a pioneering builder in AI since the early, the early 2000 and tens. So when was the moment you knew that you wanted to actually break out and start your own company?

Yeah, I mean, I think the easiest way to answer this is to tell a very, very short history of the last, what, 20 years or so of AI. So when I started getting going, I was actually initially first drawn to robotics. So I thought the idea that you could write these programs that made these physical devices do smart things in the world was like one of the coolest things possible. But way back when during that period, mid 2000s, nothing really worked. We were misidentifying horses as dogs and the Tay Chatbot came out and started insulting everybody on Twitter after 24 hours. It was all so, so early. And so I decided the right answer to do back then was to get into research. And so after multiple twists and turns because of previous work I had done in leading research oriented teams, the founding team at OpenAI brought me in to lead research and engineering there as the VP Eng. So I did that for three years and I think saw this transition from the previous era of AI of just you and a couple of pals trying research ideas independently and writing a paper, to this next world of giant scale up projects, which we did successfully with the GPTs and back then the robot hand project and Dota and all this other stuff. And after that I ended up going to Google to lead Google's giant LLM training effort. And I think what was really interesting about that was seeing the first innings of true productization on some of these models. And it just became really clear to me that first off, the recipe for building general intelligence is increasingly clear. And actually a critical part of that is having a product. Like you need a product because you need something that users interact with to teach the models to be smarter. And as a result, the dominant shape of an organization that can win at that doesn't look like what anybody had built before. It doesn't look like the standard thing of like, you know, you have researchers sitting in a corner of a building and then pitching their ideas to engineers and other products trying to get those things landed. It also doesn't look like, you know, two separate orgs with two separate roadmaps. What you really need to do is you need to go start with something. You need to start with a product shape that gets you all the way to general intelligence and you to distill that down into what research problems, what engineering problems and what product problems do you need to solve to build that product shape. Like a full end to end combination of research and product. And that sort of org structure didn't exist. And so on top of the actual technical bet we wanted to make organizationally, it was obvious we had to go do it as a startup.

Yeah, I totally agree. The last 10 years were about kind of the foundational research and infrastructure and the next 10 years about putting this into products that actually work 100%. So obviously one of the maybe naive narratives around AI, especially the type of AI that you're building, an agent that can act like a human in front of a computer, there's a narrative around AI replacing jobs and think you have a different take both on your mission for adept as well as AI's impact on the world. So maybe spend a moment on that.

Yeah, so I think what's really interesting to me about this question is I think it's fundamentally like a mission and values question. And then there's also a little bit of PRAGMATISM in there. But if you go look at how everyone has framed AI progress so far, right? Like you go read mission statements of the classic labs, it's always about replacements, about doing things that beat humans at exit, doing a why better than when, than humans, and then creating lots of economic value and then needing to figure out how that's going to be redistributed. And there's also framings about AI as being a race, right? A race of how can you get to general intelligence as fast as possible, be better than humans? And then how many companies will potentially have a seat like that and then be able to out compete each other and prevent others from all this crazy stuff? Like it's just one formulation. And I think that you can just reject this framing entirely. And I think you can reject it not only out of a sense of mission, of like, you know, hey, what if we don't want to be building systems that replace people? But also one of, but also one of just like I think potentially a misunderstanding of how technology is even diffuse within society, right? Like on the latter, when we go look at things like having a calculator or writing or using a computer, right? Those are like technologies we've built as humans over, over like the last couple of millennia that what have turned out has really happened is it's, it's, it's actually improved the cognitive skills of people. They've become cognitive technologies for people. I think this is a really interesting study that I should probably read again before I, before I quote as definitive. I think that showed that for cultures that didn't have, didn't have reading humans ability to be able to extrapolate to new events and imagine things as done by tests was a lot lower than that of cultures that had reading and writing. And so I think similarly, by having these AI systems that get smarter and smarter, that work for people instead of replacing them, we are kind of basically building a new set of cognitive technologies for people that actually end up, up leveling humans. And that's a world I'm much more excited about living in.

Yeah, I totally agree. I think people also underestimate the demand side of the equation and focus on the supply side. Right. You can automate certain tasks, but demand doesn't stay constant, right?

It doesn't, it's just, it's just changing the, it's changing the profile of work, right. Like, I feel like the real human, real human things that at least I'm excited to do and most people I know are really excited to do, they're they're excited to figure out the, like, what should we be doing? Why should we be doing it? Who do I coordinate with? And like, how do I deeply understand the people I'm working with, the people I'm selling to? Like, like, those are really human. And I think the dream is, how do you get people to focus on that and not focus on the tedium of like, ah, I have to go, like, spend eight hours shuffling things around in my database, right? Or like, I have to go like physically manually, like, create a part, right? Like those things, the execution is what we really want to be able to delegate, right?

So tell us like, what is adept?

Adept is an AI agent that helps humans do anything they need to do on a computer. And so like, let's just break down each, each, each part of that. Like, what is an AI agent? The agent term has kind of become really diluted these days because it's become the new hot thing to slap on a company. And but the actual, like, the, like, the true definition of an agent actually comes from, for folks who have been following, actually the reinforcement learning side of things. And so an agent is a, is an intelligent system that figures out the correct end actions to take to help you achieve a goal. So if my goal is, you know, I want to, I want to move these end leads from one stage to another. In Salesforce, the agent figures out, okay, here are the end actions I need to do to go. To go do that in the most efficient way possible. What an agent is not, is, is a chatbot you just talk to that just talks back at you, right? Like, it doesn't do things, it's not taking actions, right? And similarly, an agent is not a single step API call because that's like kind of the degenerate base case. It doesn't run a workflow like agents run, run workflows for people. So what Adept is doing is, is we're making it possible for every knowledge worker kind of to have this like AI teammate that they can quickly show how to do tasks on their computer and then ask the, the agent to go do there on out. And it's part of that up leveling of like human work sort of a mission that we've always had. And I think as our models get more and more intelligent today, they do a lot of things that involve, for example, shuttling data from system A to system B or helping people fill out forms or do employee onboarding or do like logistics and supply chain and all these like very operational tasks. But every incremental amount of work we put into model intelligence gets our agent to help with higher and higher level things, ultimately to the point where this truly becomes a teammate, that is a collaborator that you can talk to, interact with and solve problems together and brainstorm and also have it help you with the execution.

2024, the year of agents. I think it's a good definition of what an agent is, what it is not. So maybe break down for us the architecture of an agent in your mind from the foundational model to the orchestration layer, to the integration into enterprise data, to the UI and workflow. First of all, what's your perspective on the correct architecture of an agent and where do you think IP can be built?

Yeah, so I want to answer that for Adept and then we can answer that for the general case for Adept. We look pretty different from a lot of companies in the space because we both control our foundation model for agents and we control the agent product that enterprises use. And our company is a bet on vertical integration of those two things. So what we do is what our agent stack looks like is we have a base multimodal model that is specifically oriented around being a really good for agentic use cases. So it has capabilities that that others who are just using GPT or CLAUDE or anything like that can't get. So example of that is we're extremely good at fine grained understanding of user interfaces. So our ability for example, to figure out what do you interact with to get a task done is in the 90s, whereas when we benched that of GPT and Gemini, they're between 2 to 4 to 15% accurate. So it's like a giant gap while at the same time on the basic understanding of knowledge work data. We specialize our models towards knowledge work because that's what people use in the enterprise. We care a lot less about cat and dog photos and all the other stuff people put in these models these days and on knowledge work tasks. Actually, even though our models are pretty small and therefore very fast, they have higher accuracy than GPTV and Claude 3 Opus and Gemini 1.5 Pro. So that's like what we start with. We start with this base model. It's really smart at being an agent that's also very fast. And what we then do is, I think this is also another thing that people in the space may not have fully realized. The dream for agents isn't a giant text box in the sky that you're like, hey, I want you to go do this business task for me. Figure it out. What we learned is actually the most important thing for building useful agents in enterprise is the ability to, is the ability to be handed kind of like a standard set of operating procedures and be trusted to go do that. So we care a lot about the ability for the agent to follow any constraint on instructions going forward. What our stack looks like is that next instruction following layer followed by a user interface that makes it possible for humans to easily have oversight of the agent's behavior. The UX for these things, as we were chatting about beforehand, is not going to be that of a chatbot. Like this idea that humans want to be able to specify everything down to the T in just words, we've actually found to be quite a big limiter on productivity.

Yeah, that makes sense. So this is everyone's favorite debate in AI is who's going to win? You know, is it going to be one large model to rule them all? Is it going to be open source, fine tuned models? Is it going to be large models trained on specific use cases? Like for the agentic use case. Obviously you have a very opinion and point of view for adept, but what's your take on how the space evolves in terms of the number of large models available and what type of use cases each one is focused on and in what cases for product builders it makes more sense to just use GPT4 out of the box and focus on other areas versus fine tuning.

So I think there's a tremendous amount of fog of war. I have a fairly strong opinion but I think that new information I think could change a lot of this. My view right now is that, my view right now is that there's a crop of companies, right like that are training, that are just in the business of training models. Right? So like the OpenAI's with. Well, OpenAI is not quite right because they also have their own products. But like the just the GPT training part of it anthropic is entirely around training models, Cohere, Mistral, et cetera. I think that that space is really quickly commoditizing because it's the same corpuses of training data. The architectures are mostly similar. There's no real, there's no real long term defensibility in ideas in that space because they diffuse within order of months and as a result it really just becomes a cost to capital game. And I think that there will be end organizations that will be able to afford that. Open source is getting really good really fast. Meta is obviously investing a lot in the space really smartly for them, but that also just sets the floor if you've got a model that isn't as good as the next Llama, like what, you don't really exist as a business. You don't get to charge more than the cost of compute. Right. And so I think companies in that space will have to look for alternate ways of making money besides just having better models. But I think that, like, I think that long term what's going to happen is that like, is that people who figure out how to hook up very differentiated approaches of generating data for the most valuable tasks to the base models that they also control will have tremendous leverage. And I think that's why I remain bullish on OpenAI. I feel like the fact that they have ChatGPT, that they have a lot of developers using ChatGPT actually for code use cases, all of these things gives them a flywheel where they could overtake someone who's literally downloading more stuff on the Internet. Whereas an API business doesn't give you an opportunity to have a data flywheel.

Yeah, I think that's, that's really smart. And so tell us a little bit about how you're designing the front end of Adept, the agent, in order to maximize valuable data collection.

Yeah, so I think just, I think it's a really good question because that's exactly like, like our bet is that by end to end optimizing our foundation model with the agent use case, we're collecting agent data that makes, that makes the, that makes the whole thing smarter. I think that the way that I think about it is there's two parts to it. One of them is like, one of them is like, why does this stuff make the model smarter? I think when we go look at, when we talk to ChatGPT or Stuff, there's been so much RLHF data. When you basically take human feedback on how well a model did something to improve the model's behavior through a reinforcement learning or something that looks like reinforcement learning sort of loop. Like, it's like these models out of the box are trained to just clone human behavior so they don't understand reward. You give it a reward signal by collecting lots of data about what good and bad looks like and you teach them all to follow the good. So that's RLHF. And it's been with things like ChatGPT, a lot has been done on it in the summarization, chat, all those other spaces. Our goal with Adept is to do that same loop, but with agents. And that's so important because out of the box these foundation models are really unreliable for agent tasks. And reliability is almost the only thing that people care about. You go ask them to book a flight for you in the background. Even for a consumer use case, which we don't cover, you expect that to do a good job and not to have already swiped your credit card for something entirely different on the wrong date.

What are some interesting techniques of how to make RLHF actually productive? Because I've also heard stories of, you know, after a model's out in the wild, you know, and they get human, human feedback. Parts of it can actually get worse because, for example, humans are bad at probability or bad at math.

Yeah. So basically I think the answer is you want. You just want to be. This is actually a nice thing of our enterprise strategy is you want to be learning from the. You want to be learning from the smartest knowledge workers in the world. I think that like collecting more RLHF data for how to chit chat about your day doesn't make your model smarter. But in a work setting all these people are like, we're all paid every day to try to use intelligence as a way to get an advantage in business. And so the data that comes from people doing that is just inherently more valuable and less prone to some of the same problems you were talking about. Thank you. Arbitrary chit chat data that makes sense.

And what's your take on how vertical specific agents will become right. Is Adept going to be my accountant, my lawyer, my analyst, my researcher, et cetera or where do you think lines will be drawn?

I think that my boring take on this is that within a couple years the whole AI concept is going to fade in the background and we'll just consider all companies will be AI companies, so it just won't even matter. And it's just going to be like a lot of Amazing Enterprise SaaS, businesses built on AI agent technology that each has found a niche and is like beating each other on things like go to market and understanding the customer. And less on is my agent 5% smarter? I think there will be a lot of companies that look like that that'll cover like head use cases like invoice processing. Right like, or like customer support even. Right. And so there will be many great companies built that way. But I think the reason why we're really bullish on the path we're taking on generality with Adept is, is that if you just look under the hood a little bit, even some of the most obvious use cases in enterprises that of workflows people want to do sound like they should be really common. Right. Like, but are extremely customized to that business and their customers. And so, and so like if we were to a portion, you know, of the overall pie of agent tasks to be done, maybe like, like 10% of it or less. Right. Is cookie cutter and 90% is like, I need to teach my specific workflow to the agent and we want to eat that 90%. Yeah.

I feel like you were several years ahead of kind of where the market is.

Right.

Where everyone, everyone's been enamored to your point around chatbots and image generation, video generation, but it feels like the real value is going to come from actually executing work. And that's actually a different problem.

Yeah. And our job is to execute custom work basically.

And so on that point, again, being an independent thinker here, I would say a lot of people, including OpenAI have had success by just launching products into the wild. There's this magic effect where if you're first to creating an agent that really works and Adept is obviously one of the very few companies in that category, just releasing it to the wild can create a lot of Twitter buzz. You've taken a different approach on go to market. You've decided to go enterprise first. Talk us through your thought process.

Yeah, so we're extremely quiet as a company. I think we've been really focused on building something that works well, that is reliable enough to be deployed and then getting that in the hands of customers rather than sort of a bottom up strategy combined with a lot of marketing. And some days I actually wonder whether we did the right thing. But what took us down the path of focusing on enterprise actually comes from a lesson that we learned last year, which is that for agents, the only thing people care about is reliability. When you talk to a chatbot and the chatbot says something dumb one out of three times, you don't care. Right. Because it's like you either enjoyed the interaction or help stoke some thinking or something like that. But if you're trusting this thing to go handle shuffling data around in Salesforce and it deletes a third of your records, you're never going to use this thing again, or at minimum it's not useful to you. Right. You could have just done the work. You could have just done the work yourself. And so we realized that the only thing that customers cared about was reliability. And because of that we decided to focus on enterprise because there's a lot of value there and in those settings we can control for very high reliability and get stuff out in the market. Now we could Put out more toys like we put out a toy called Experiments that was just for kicks end of last year. But I feel like that's not the, that's not the path to glory.

In the space and in this stack of building adapt, you're solving a lot of hard problems, right? The foundational model that you're building yourself, the orchestration agentic use cases, the UI and workflow. How big of a component is the actual enterprise integrations as part of that stack in terms of how your engineering team is spending your time and how difficult that problem is?

It's super difficult. I think the other thing that I think most people won't say directly in AI is that very little of it is glamorous. A lot of it is about even on the model side it's like data, low level systems issues, all that stuff. But on the customer side too, it's like you just got to do whatever it takes to get a deployment that is reliable enough for those folks to depend on at work. One of our use cases literally involves a physical truck being sent to a, a shipping container port. And if we screw up along the way, there's a truck being sent with no container on the other end. So that's really bad. And so what we end up doing is right now we work with the customer to get the reliability. But our whole like a large part of our research roadmap is how do we make it more and more and more out of the box? 95% from the get go, as you're.

Building this business, how do you approach the potential need or pull to do custom integrations or service oriented work to make it work for certain company versus the long term vision of a generalizable agent.

I think this starts from a belief that we have that actually is informed by the machine learning side, which is that the best thing to do is to figure out how you can delegate generalization to the neural network. So how can we try to make as many customer use cases, ones that sort of inform the ability of the base model to serve customer? N +1 looks like an interpolation of two existing customers you already have. And so because of that our philosophy is like everything should be general unless you have to ship this thing tomorrow and then we'll go build a specific thing there. But we'll add that that customer's work in a, in a, in a dummy data fashion to our evaluation set and then be like all right research team, how do we make sure out of the box the model can do well for that customer and Then over time, rip out the custom bits that we've built.

Yeah, super interesting. So maybe let's talk about the future for a second. Humans are going to focus on their customers, focus on deciding what needs to be built, not actually building the thing. So in this world, what does software look like? There's systems of records, there's agents, there's legacy apps, maybe there's new types of apps. Walk us through what this future agent world looks like.

Yeah, so that's a really astute question. And it's part of why for us at Adept, like, we have really emphasized the importance of design. From day zero. We've hired a lot of creative technology to shape people, in large part because we knew that the steady state of agents is going to look like a reinvention of how you use your computer. And how you figure out what that looks like has a tremendous influence on what your product shape is and also your actual modeling problems. So it's all one giant game of co design. My view basically is that computers have always been about giving leverage to people. And it started out with very little leverage. You had a punch card, you were literally punching in programs that programs didn't do very much. And then you had the command line, which gave you. Which was an interface abstraction that gave you way more leverage on your time and ability to do things. And then when we realized that we could give people even more affordances through graphical user interfaces, we transitioned over to mostly doing that. With the exception of certain specialized tasks. We dropped back in, into the command line. What that's really done is that for every unit of energy as a human, you can do way more in your computer you could ever do before. I think what's really powerful about agents is it's the obvious next step. Beyond that, once agents exist that can control your machine, you only need to drop into the GUI to go do things that for some reason the agent can't do for you. That's something you want to supervise. Right. And so I think to some extent, like my analogy is like, you know, when Windows 3.1, that Windows 3.1 era, your computer boots up into DOS and you type Windows, you hit Enter, and then you get into the gui. I think we're in the very early innings of that transition for agents right now. And I think that ultimately what it'll likely look like is that you become, you become a coordinator interfacer with an agent or a set of agents on your machine that you can basically work with. And it's almost like a generative UI way The agent should be able to generate the right affordances for you to best collaborate with it on any particular task and everything else will be abstracted away.

Yeah, I'm very excited for that feature and thank you for building it. We one final question on that. So in this world where we have, you know, several billion agents, right, you know, maybe several agents per person, what are the missing infrastructure pieces for agents to operate in that world? I'm thinking how do agents pay each other? How do they, you know, pass a captcha test? How do they understand how to, you know, data privacy, what to share with others? What needs to be built to allow agents to operate in this world?

That's a really good question. I think the most interesting thing about agents is that especially if you take the adept formulation, which is that it uses your computer like a human or can do that in addition to APIs, you kind of get access to the same rails underneath that we already have as people. So payments can be handled via regular old payment channels. Right? Like we have to solve credentialing. Like how do you, how do you safely share creds with your agent and have it be able to run either locally or in a VM or something like that? Like that's going to be tough. But you get to use everything else just like how things are today. I think what's going to be really challenging is that I think the sheer amount of compute that is going to go to training and serving these agents is going to be colossal. And I think what a big shift that's going to happen is, is when you are literally getting hours of knowledge work that you would have had to do yourself done by your agent. Your willingness to pay for ridiculous inference is enormous. And so I think what we're going to see is we're going to see people. I think it all again boils down to hardware. I think we're going to see people invest in ridiculous data centers, not just for running small edge models, for actually running the smartest models possible on the planet. And I don't think we're anywhere close to saturating that.

So I'm going to buy some more Nvidia stock.

Yeah, I would probably do that. Okay.

David, thank you so much for taking the time. Obviously we're lucky and privileged to be investors in Adept. And thank you for continuing to push this industry forward.

Thanks Seth. This was awesome.

Thanks for listening to product LED AI. You can find more information about today's interview and the entire series on the website productledaipod.com you can subscribe to the show on all major podcast platforms and watch the video version of this interview on YouTube. And if you want to get all the links and details delivered right to you, sign up for my LinkedIn newsletter. I'm Seth Rosenberg, and this is product led AI.

Sat.