Getting the Most From AI With Multiple Custom Agents ft Dust’s Gabriel Hubert and Stanislas Polu

We've asked the entire world to move from calculator technology, punch the same keys, you'll get the same result, to stochastic technology. Ask the same question, you'll get a slightly different result. This has not happened. This is the biggest shift in, you know, the use of the tools that we have since the advent of the computer. We're asking an entire cohort of the workforce to move to a stochastic mindset. And the only way you get that is by having a risk-reward ratio and that you're comfortable enough. It's like, You know what? I don't, I'm not asking it to be right 100% of the time. I'm asking it to give me a draft that saves me time many, many, many times over. And that distribution of ROI is something that I'm comfortable exploring with and iterating on. And I think that that is really one of the predictors that we see in people who've tried ChatGPT or in people who are just curious with new technology is they expect that some of it's gonna be a bit broken, but the upside scenario to them is so clear and so 10x that they're willing to make that trade-off or that local risk. To, uh, to get things started.

Welcome to Training Data. This week we welcome Gabriel Hubert and Stanislas Polu, the co-founders of Dust, a unified product to build, share, and deploy personalized AI assistants. At work. Founded in early 2023 after spending years at Stripe and OpenAI, second-time founders Gabe and Stan started Dust with the view that one model will not rule them all, and that multimodal integration will be key to getting the most value out of AI assistants. They were early to be convinced that access to the proprietary data you have in data silos will be key to unlocking the full power of AI. And they know that you want to keep that data private. We've worked together for 18 months and their predictions have been consistently prescient. So today we decided to ask them about those predictions. We'll get into their perspective on how they see the model landscape evolving, on the importance of product focus over building proprietary models, and on how AI can augment rather than replace human capabilities. Stanislas Polu, Gabriel, welcome to Trading Data.

Thank you. Glad to be here.

Yeah, thanks, Constantine. Super happy to be here.

Guys, first thing that I want to ask is you started this company in early 2023. At the time, it seemed like one model might rule them all. Uh, and that model at the time was, I think it was probably GPT-3.5. I don't know if 4 had yet come out, but that was way ahead of the curve and people were super blown away. You guys came out with a pretty contrarian view that there actually would be many models and that the ability to stitch those together and do advanced workflows on top of that would be important. So far you've been completely right. How did you get the confidence to make that decision a year and a half ago?

Yeah, I think, uh, on the, on the model part, it's, uh, it was clear that, um, many labs were already emerging. It was not clear from the kind of general audience, but for the people people that knew the dynamics of the market was called Minilabs, who were emerging. And I think it was kind of natural to us that there would be competition in that space. And as a result, there would be value in enabling people to quickly switch from one model to another to get the best value depending on their use cases.

Yeah. And I think from the user standpoint, the point on being able to quickly evaluate and compare is obviously important. Looking ahead, or already at some of the conversations we're having, it seems that the levels of scrutiny, security, sensitivity of the data that's being processed may also influence some different use cases. And so we're excitingly seeing people thinking about running smaller models on device for some use cases. And you could imagine a world where you want to be able to switch between an API call to a Frontier model for something that's less sensitive, absolutely crucial to get like cutting-edge reasoning capabilities for, and some smaller classification or summarization efforts that could be done locally while the interface that you use for your agent or your assistant remains the same. And that switching sort of requires the ability to have a layer on top of the models.

So you guys have been right about this every time, as you've called this out. And so many of your predictions over the past couple of years of partnership have been non-obvious and then correct. Uh, I think this is still not obvious, as in there will be many models. You'll have some local models, you'll have some API call, and then you actually, as a customer, want to choose between them or want to have some control. First of all, why do you think that will be? As in, why will there be multiple models? Secondly, why doesn't that get abstracted away by some sort of router mechanism, some hypervisor layer? And does that happen? And would you be that supervisory layer? Yeah, help me understand that.

I think there's really two modes of operation if we think about the future. So it's a bimodal distribution basically of the future. There's one where the technology as it stands today keeps progressing rapidly, in which case there's still going to be competition from rather big labs because there's going to be incredible need for GPUs to build those larger and larger models because the only way we know to get make those models better today is mostly by scale. In that world, this kind of dynamic of being able to switch to the best model at time t will remain true for a long time, I guess, until we reach whatever it is that is at the end of that dynamic. And then there's the hypothesis, and we can talk about that later in more details, but the hypothesis of maybe the technology plateauing, in which case it's not going to be one model, it's going to be a Gaussian model and eventually everybody will get this model and eventually on your MacBook M6, you'll be able to train a GPT-6 in a few hours in a couple of years. And then the kind of router, the router need, those kind of disappears because the technology is really commoditized in terms of just producing the tokens and every company will have their own model. We would have our own model in that world.

Well, we got to push you on that. So there's, you're building a business where you sort of win regardless of which one of those worlds we go into. Which one of those worlds do you think we're going into?

This one is definitely tricky. I mean, it's interesting because, so in terms of capabilities of the models, we've seen or had the perception that the ecosystem was moving very quickly over the past 2 years. We've seen larger context, support for audio, support for image and stuff. But at the same time, the one core thing that matters for changing the world is the reasoning capabilities of those models, right? And the current reasoning capabilities of those models has been actually pretty flat over the past 2 years. They are at the level of GPT-4 as the end of its training, which is roughly slightly over 2 years ago, if I remember correctly, for the end of the internal training at OpenAI. And so that means that over the past 2 years, in terms of reasoning capabilities, it's been somewhat flat.

Well, so hang on. So there's the Kevin Scott point of view, which is there's actually exponential progress, but you only get to sample that progress every so often. And so in the absence of a recent sample, people interpret it as having been flat when in fact there's just sample bias. You can't see it. So do you think he's right? Do you think there's actually exponential progress, we just haven't gotten to see it yet? Or do you think it's actually asymptoting and reasoning breakthroughs have not progressed at the rate one might hope?

As far as I'm concerned, I have a strong feeling that it hasn't been moving as fast as I would have expected in my most optimistic views of the technology. And so that's why I'm allowing myself to ask the question or simply consider the different scenarios.

What I think one of your predictions for 2024 also is that we would have a major reasoning breakthrough. Do you think it's coming?

Yeah, it's going to be a tough one because this one hasn't come for sure. And even GPT-5 or GPT-N+1 or Claude-N+1, doesn't matter who cracks that first, hasn't come yet. And there's many reasons to believe it might not be a core technological limitation. You can make many hypotheses as to why it might be the case that it takes time. The scale of the clusters required to train the next generation of model is humongous and it involves a lot of complexity from an infrastructure and really programming standpoints because GPU fails when you scale to that many GPUs per cluster. They fail pretty much all the time. All the training is very synchronous across the cluster. And so it might just be the case that scaling up to the next order of magnitude of GPUs needed is just very, very, very hard. And that wouldn't be kind of an inherent limitation. It's just a phase where we learn how to go from Red 1 to Red 5, but for GPUs, basically.

Stan, you were at OpenAI at a pretty critical point in time. So, People know you from the DUST experience, but one thing that has to be remembered about Stan is you were a critical researcher at OpenAI from 2019 through late 2022. You got a bunch of wonderful publications. Some of them relate to mathematics and AI. You worked on these with Ilya Sutskever and the crew at OpenAI. Do you think that mathematics will be essential to this type of reasoning breakthrough, or is it orthogonal? That's something that we're actually going to learn on textual or language data.

I remain quite convinced that it's a great environment to study. And it was a thesis that we had at the time with Guillaume Lempereur, who then founded Mistral. He was working at FAIR on exactly the same subjects. And our motivation was really shared at the time. We really were frenemies competing in the workspace, but really friends by the ideas. I think the idea there was that mathematics, and in particular in its form of formal mathematics that gives you perfect verification, is a very unique environment to study reasoning capabilities and to push reasoning capabilities because you have a verifier. So you're not constrained by being able to verify the prediction of the model that in an informal setup would require humans checking them. To some extent. And so that very bit is probably something that has to unlock something at some point. It hasn't yet for many reasons, but at some point it should unlock something. So I remain extremely bullish on the kind of math and formal math and LM studies.

Yeah, I remember one of the ways you were presenting it to me when I was still very much ramping up was, maths is the door to software, software is the door to the rest, and starting with some of the critical systems that were the very only ones to have been hand-proven and hand-verified as an example of how much more costly it was to do it by hand than do it by machine. Uh, and, and an indication of the future gains we could expect from being able to extend that and democratize that.

Uh, you guys see a lot of action through the DUST API calls. When you build a DUST assistant, you're able to choose what type of underlying model to use. Uh, you're able to call many different models. Like me as a user, I often call not just Claude 3, but I call GPT-4 and I call the DUST assistant and I call, uh, in my custom assistants, I select one of many options. What have you guys seen in terms of trends? What's performing really well? I've personally been super impressed by the Anthropic models as of late. Um, but you guys have a much closer view of that.

I think the, um, I mean, so a word of caveat on On trends, you know, you're going to have the usual cognitive biases. The grass is always greener. People are going to want to switch just to see what it looks like on the other side. And so when you're observing those switches, you're not necessarily observing a conviction that the bottle on the other side is better. You're just, you're observing the conviction people want to try. Um, but it is true. We've gotten great feedback on, on Claude's latest Sonnet release. Um, and empirically we're seeing some stickiness on, on, on that model. in, in our user base. I think that, uh, you know, word on the street is, uh, for some coding application, CodeStrile is actually performing very, very well.

We haven't, uh, yet made it available, uh, through Dust, uh, but we're just today.

Ah, there we go.

Sorry.

See, this is the thing you get for being in San Francisco and waking up to do reporting at 7 o'clock in the morning. So yeah, CodeStrile apparently is, uh, is really interesting on some, on some coding capabilities. And then you have to mix it in with the actual experience that people are getting. So reasoning cannot be fully made independent from latency. Latency at some points last year could be basically a way to tell the time in San Francisco. You could see latency literally in the API as people were waking up on the West Coast. So people have use cases that may be more or less tolerant of those. So we cover the Gemini models, Anthropic's models, OpenAI's, and Mistral right now. We have seen some interest in moving away from the default, which when we first launched were OpenAI's models. Not to say that 4o isn't performing very, very well.

Over the past year, there's been a lot of enthusiasm about open-source models. It's actually one of your predictions. Stan, you have these great predictions every year about AI. I always really enjoy Reading them, one of them was that at some point this year, an open source model will take the brief lead, uh, for LLM quality. Uh, that doesn't seem to have happened yet. And it also seems like the enthusiasm, uh, around, not the enthusiasm around, but rather the lead slash acceleration of, uh, the open source models in comparison to the closed source models has maybe slowed down a little bit, maybe back to that Kevin Scott point. About we're sampling at discrete times as opposed to continuous times, we just haven't seen it yet. But where do you think the open source ecosystem is going to go? Will it actually at some point surpass the closed source ecosystem?

I mean, that remains that, that, that, that echoes with what we said earlier. It's really in that bimodal distribution. There's one distribution where open source goes nowhere and there's one distribution where open source wins the whole thing, right? Because if the technology plateaus, open source obviously catches up and eventually everybody can train their, their high quality model themselves. And at that point, there is no value in going for a proprietary model. So I think there's a scenario where open source really is the winner at the end, which would be a fun turn of events, obviously. And then in the current dynamic, it's true that open source has been lagging behind so far. Obviously, there's, I think the one that has to be called out is really Facebook or Meta efforts, because they have what it takes to train an excellent model. And so far, there's been releasing every model very openly. And so that's exciting to see what will come out of them in those next 4 months to maybe make the prediction true. The caveat to that is that assuming the best models are the largest, which is a somewhat safe assumption, yet it can be discussed, it means that that model will be humongous to some extent. And so that means that even if it's open source, nobody will be able to make it run, right? It'll just cost too much money. You'll need 8 GPUs just to do inference. And so that will really trump the kind of usage of those models, even if they're better in the current state of affairs in terms of costs of running them.

It's a point for consumption that's interesting because that means that you might still have a world where there's a lot of API-based Inference demand for API-based inference, regardless of whether the model on the other end is controlled, hosted, open weights, whatever, um, just because of the technical abilities to perform it.

One of your founding assumptions kind of related to model quality and model performance, and this goes back almost 2 years now, was that even as of 2 years ago, the models were powerful enough and potentially economically viable enough that you could unlock a huge range of unique and compelling applications on top, and that the bottleneck, even at that point, was not necessarily model quality so much as product and engineering that can happen on top of the model. I don't know if that's a consensus point of view today. You know, we still hear a lot of people who are sort of waiting for the models to get better. For what it's worth, we happen to agree with you, but the question is, What did you see in 2022 that gave you that point of view? And if we fast forward to today, what has your lived experience been deploying this stuff into the enterprise in terms of where are the product and engineering unlocks that need to happen to bring this stuff to fruition?

My triggering point for leaving OpenAI was seeing and playing with GPT-4, and it is, it was coming from two very contradictory motivations. The first was, I see GPT-4, it is crazy useful. Nobody knows about it. Nobody can use it yet. And still it exists. And literally it's almost already in the API. I mean, at the time it was GPT-3.5 in the API, which was kind of a slightly smaller version of GPT-4, but on the same training data. It was a crazy good model, which was, and it was basically Codex, the the base model, and it was much better than ChatGPT, and it was available in the API. And yet the ARR of OpenAI was ridiculously small at the time, like in existence by all standards of what we see today. And so that was kind of the motivation. And that was mixed with the fact that I was starting to feel the I mean, I had the intuition that it would be hard to invent an artificial mathematician with the current technology. And so I was kind of seeing it as not a dead end, but a very long path, slow path forward on what I was working on. And at the same time, I was seeing the utility of those models already when you use them for your day-to-day tasks. And so that was the first motivation. And the very contradictory motivation that I shared with Gabriel, at the time was, if that technology goes all the way to AGI, it's the last train to build a company. So we better do it right now because otherwise next time it's going to be machines. And I absolutely didn't answer your question, but I'll let Gabriel answer your question.

I think what got me excited and when we did start brainstorming on the ways to deploy this just raw capability in the world, what where it made sense to dig was one insight on some of the limitations of the hype around fine-tuning at the time. People were talking a lot about fine-tuning. A lot of consultancy firms were selling a lot of slides that were essentially telling big companies to spend a lot of money fine-tuning. And the two things that cut it for me was Dan saying, you know, one, it's expensive and you do it regularly and nobody knows that they'll have to do it regularly. And two, it's really not the right idea for most of the things people are excited to fine-tune on. And in particular, fine-tuning on your company's data is a bad idea, as opposed to maybe sometimes fine-tuning on some specific tasks where you can see gains. But the idea that bringing the context of a company, which is obviously every real company's obsession, like how does this work for me? How do I get it to work the way I like it to work? Was going to happen with technologies that weren't just changing the model itself, but rather controlling the data it has access to, controlling the data any of its users have access to. And those are somewhat hybrid models between new world and old world. The very old world version of it is, you know, the key holders are still the same. The CISO is the one deciding how new technology is exposed to members of a company, the guardrails that are in place, the observability that's available to the teams to measure its impact and any data leaks. Those are old software problems, but they still need to be rolled out on very new interfaces. Because the interfaces now are these assistants, these agents. And then some of the new problems are around access controls. Does access controls look and feel the same in a world where you have half of the actions done by non-humans? Now, I might want to have access to a file that's like 2020. Do I have access to the file? Yes or no? In 2024, it's like, well, maybe an assistant might have access to the file and can give me a summary of it that leaves out some of the critical information I should not have access to, but still gives me access to some of the decision points that are important for me to move on with my job. And that set of primitives, that set of nuances just doesn't really exist in how documents are stored today. So if you think about deploying the capability in a real-world environment where people are still gonna have to face those controls and those guardrails, the product layer is actually very thick. The application layer to build the logic and the usability to ensure performance, but also adoption is, is, is quite thick. And that was the, I think that was the go to say, all right, there's a lot to do here. We might get started.

Maybe you can dig into that because when we intersected in Q2, 2023, Q1, Q2, 2023, a lot of people were still starting these foundation model companies. And you guys had a very specific opinion, which is the future is application layer. And there's going to be a lot going on under the hood. And we're just going to be an abstraction layer on top of that and let things happen as it see, as it happens. We're going to succeed in any case by building something that people actually use and love. First, how do you have the conviction for that? Secondly, how has that been playing out? What has been the hard part about it? You mentioned the CISOs and the enterprise and enterprise deployments. Uh, you guys have been way ahead of the curve on RAG. I mean, everyone was talking about fine-tuning, but you guys have done so much in terms of, uh, retrieving— it was just before it was even called that really. Retrieving and actually making smart decisions around information. Walk us through the step-by-step of, from the idea of application layer to where you are today.

You can imagine the application layer conviction existing in a world where you still decide to build a frontier model. The reason we split those two is one, it seemed like a lot of money for a lot of risk. And I mean, a lot of money for a lot of risk to try and develop a frontier model or an equivalent to a frontier model. And also make a bet on the way it was going to be distributed. And, um, it's— so our internal slogan was no GPUs before PMF. Like, we don't see the value in training our own model until we actually know which use cases it's going to get deployed on. Uh, and there are much cheaper ways to explore and confirm which use cases are actually going to make most of value and generate most of the engagement. The second reason was really about this data contradiction. The fact that the cutoff dates for training on internet data are hard to set continuously, the fact that you can't actually get an internal understanding of what happened last week in a frontier model means that fine-tuning is a hard problem, that it is not a solved problem at scale. If you walk from that conviction backwards, that means it's then there are many cases where it's not solved. So another technology has to be the one to deliver most of the gains. And extracting a small piece of context from documents where it lives, feeding it into the scenario, the workflow that you need help for. The one trend that seemed interesting was that actually many decisions require limited amounts of context and information to be greatly improved. So the context windows at the time that were small were already compatible with some scenarios of saying, let's just bring the information in. And what we've seen over the last year, of course, is the increase in size of those context windows, which just makes it easier to expose all the right data, no more than the right data, hopefully, to the reasoning capabilities of the Frontier model. And what we've experienced is, first of all, it takes time for people to understand those distinctions. It's hard. And that you have to get yourself out of your own bubble regularly to realize that it's true, the world, the future isn't quite evenly distributed yet. And people have varying assumptions on what it means to roll out AI internally or roll out the capabilities of these frontier models on their workflows. And you have to walk them back on what they really care about, which is always very simple things. I want to work faster. I want to know the stuff that I'm missing out on. I want to be more productive or more efficient in some tasks that I find repetitive. Then only bring the explanation of what technology is going to solve that when it's absolutely necessary, because people will worry about their experience and how they feel about it more than how it's working under the hood 99% of the time. Uh, the, the big insight that's happened and that I think we are leaning into, we have been for a while, and it's great to see some of the market also doing that is people are actually really good at recognizing which tool they need in the toolbox. Like I think we've, we've, we've not respected users enough in saying, you know, you need a single user that does absolutely everything and the routing problem should be completely abstracted from you. You should ask this question to the one oracle and the oracle will reply. People are pretty comfortable telling a screwdriver from a hammer. And when they want to get to work and they need a screwdriver, they're very, very disappointed when what they get is a hammer and it sounds like a hammer response. And so specializing agents, specializing assistants, and making that easy to do, design, deploy, monitor, iterate on, improve, all those verbs that require product service, it was it was quickly apparent to us that people were very comfortable with that. And so the number one question that made us feel like we had an insight to hang on to and lean in on was everybody asking us about Dusk was obsessed with the top use case. It's like, what are people using it most for? What is the top use case across companies? And I could almost see the Amazon eyes trying to decide which diapers.com they're going to verticalize and integrate, like which verticalized use case should we now just build as a specialized version of this? But I think the full story is fragmentation. I think the story is like giving the tools to a team or to a company to see opportunities for workflows to be improved on, augmented, and understanding the Lego bricks that are going to help them do that. So rather than encapsulate the technological bricks that are useful and abstract them away from users, exposing them at the right level gives people a ton more autonomy. And really the ability to design things that we had never thought of. Some of the scenarios that have come up, we, we literally could not have imagined ourselves.

That idea makes sense, like the fragmentation and providing people with the Lego blocks to see what sort of use cases emerge. Just to make it a little bit real though, can you share a couple of use cases that you've seen in your customer base that have been unique or surprising or particularly valuable? Just something to make it a little more tangible.

There's obviously a ton that people are thinking about, like I was also, I'll start the, the category of obvious use cases that have been interestingly, uh, and quickly deployed. Are enablement of sales teams, support teams, marketing teams. And that is essentially context retrieval and content generation. I need to answer a ticket. I need to understand what the answer to the ticket is and generate a draft to the ticket. I need to talk to a customer. I need to understand which vertical they're in and how our product solves their problems and draft an email to follow up on their objections. I need to prepare a blog post to show how we're differentiated from the market. Again, like I'm going to go and plow into what makes us special and generate with our tone of voice. Those were pretty obvious and quite expected. What I've been excited by is to see two types of things. One, very individual assistants, personal coaches, people generally actually quite young people in their first years of career asking for advice on a weekly, on a daily basis. Like, how did I do today versus my goals? Where do you think I should focus my attention in the coming days? Can you actually break down my interactions on Slack and in Notion over the past couple of days and say where I could have been more concise? I'm getting the feedback that I'm sometimes talking too theoretically. Can you point out the ways in which I can improve on that in these two notes that I'm about to send? And so that's exciting because our bet was, you know, we want to make everybody a builder. We want to make everybody able to see that it's not that hard to get started. And by reducing the activation energy there, to see small gains immediately rather than wait for the next model or the next version that's going to really solve everything for them. Personal use cases have been great for that. The second family of use cases that I'm excited by are essentially cross-functional. So where the data silos exist because the functions don't speak the same, they speak the same language, but they don't speak the same language. And so understanding what's happened in the codebase when you don't know how to code, is powerful. Having an assistant translate into plain English what the last pull request that's been merged does is powerful. It's powerful to people that were blocked in their work, didn't know who they should bug to actually get an update. So, you know, marketing to engineering, sales to engineering. The other scenarios are, you know, extracting technical information from a long sales call is powerful because it means that the engineer doesn't need the abstraction of a PMM or a PM to get nuggets from the last call with a key account. They can just actually focus the attention of an assistant on that type of content on their own project and get those updates. So I'd say that's the family of assistants that we're excited by because they really represent, I think, the future of how we'd love fast-moving, well-performing companies to work, where the data that is useful to you and the decisions you should make is always accessible. You don't need to worry about which function, decided on it or created it, you can access it. And that fluidity of information flowing through the company helps you make better and faster decisions day in, day out. Yeah. Any other examples that I'm missing, Stan, that you think you're excited by?

No, I think what I wanted to add is the fact that, as you said, the usage is extremely fragmented. We see over and over the same scenario. And so we have data to back that kind of proposition is As we, we built Dust as a sandbox, which is, makes it extremely powerful, uh, and extremely flexible, but also has the complexity of making activation of our users, uh, not trivial. Because when you have an horizontal sandbox-like product, you're like, yes, but for what? And so generally the pilot phase that goes with our users starts by clearly identified use cases. So they really kind of try to answer the question, what are the use cases I should care about? For my company and try to identify a couple of them. And we always see the same pattern. We see first use case gets deployed, usage starts. We try to move laterally to another use case. Second use case gets deployed, usage picks up a little bit more. And then we generally go through a phase where the usage is kind of flat, increasing slowly. And eventually it reaches kind of a critical mass of usage. And all of a sudden it skyrockets to something like 70% of the company. And that's kind of the pattern of activation of our users. And the skyrocketing to 70%, the usage picks up to Tom. The original use case that were identified by the stakeholders become just anecdotal compared to the rest of the usage. And that's where we feel like Dusk provides all its value. And it's very hard to know for us what are all those use cases because we have examples of companies with a few hundred people and a few hundred assistants. And so it's just, it's just hard to answer the question, what are the best use cases?

Like, that's, those are great examples. And that, that calls to mind an analogy that I would like to try out on you guys. And you may puke on this analogy, but this is, this is what just showed up in my brain, which was, um, a lot of those use cases you described, you could imagine some sort of vertical application being built around those use cases. And the analogy that comes to mind is There are a gazillion vertical applications, and yet where does a lot of work happen? Spreadsheets. Why does it happen in spreadsheets? Everybody knows how to use a spreadsheet. They're there, they're flexible. You can, you can customize 'em to your heart's content. And so the analogy that I'm wondering about is this almost like the spreadsheet of the future? You know, some of these applications may get peeled off at a vertical specific applications, but even then people are still gonna come back to the, to the personal agent because it's just, It's there, it's available, it has access to your data, it's familiar, you know how to use it, you can build what you want quickly and simply and effectively. Like, is that a reasonable analogy for what this kind of is?

I think it's an amazing analogy for another thing that I'm thinking about, which is, it took me the longest time to get Stan to use spreadsheets when we started working together. And this is way back when, this is like, I don't know if it was 20 years ago, 15 years ago. And then at one point, Stan uses it for something that is like, oh wow, this is kind of like a cool REPL interface where you can just get the results of your functions in real time. I was like, yeah, that's now the word. He's like, it's a cool REPL interface for non-engineers. I get it now. And yeah, I think it's also interesting for that. The experimentation cost is very, very low. If you think about, um, the way in which like some of our customers try and understand, try and describe the gains that they're experiencing or that they're seeing and their excitement for the future. Is some functions we've had 80% productivity gains. Some functions we're seeing 5% productivity gains, and we're not even sure that we're measuring them right. But we're seeing gains when the specialization of the assistant is close enough to the actual workflow that it is able to augment. The distribution problem of that with a verticalized set of assistants is almost impossible to solve. How are you going to get that deep into that function? At a time where budgets are tight, decision-making on which technology is going to be a fit is sometimes complicated, when sometimes that's where the performance gains are the most obvious. One of our users has seen like 8,000 hours a year shaved off two workflows for an expansion into a country where they decided not to have a full-time team. And so basically sparing you some of the boring details, but like the ability to review websites, compare them to incorporation documents in a foreign language, have a policy checker that was making a certain number of checkpoints very clear to the agents that were reviewing the accounts, all in a language and in a geography that none of these people were yet familiar with because they were really exploring the country. And immediate gains, like very, very easy iteration on the first version of the assistant, 2 weeks to launch it into production, roll it out to 3 human agents that were then assisted by these assistants. And their CTO sharing like, you know, we're seeing north of 600 hours a month. I'm thinking our pricing's terrible. But what I'm excited by is that that case could not have been explored or discovered with a verticalized sales motion because I just don't know how you get to that fairly junior person in a specific team and actually are able to pitch them and deploy that quickly. Whereas if you have that common infrastructure that people understand the bricks of, not everybody knows how to do some products, not everybody knows how to do a pivot table, but everybody understands that they can just play around with the basic things and probably get help from somebody close to them. That's the other thing we've seen, you know, um, the, the, the map of builders within companies, this heat map of people. What's amazing about it is that it's people who are just excited about iterating, exploring, and, and testing new stuff, which I think correlates well to, you know, high performance or high I think Dust is heat-seeking for potential and talent across your teams because the people using it the most are people who are the most comfortable saying, I don't feel threatened by something that's going to take the boring and repetitive side of my job away from me. I'm excited to have that go away and focus on the high-value tasks.

I think for the first 6 months, I was one of the loudest voices saying, what is that main use case? I think you guys heard many, many times. And then eventually I realized, like, this is a primitive— we're talking about spreadsheets. You could talk about, frankly, a Word document. You could talk about Office Suite. When I interface with Dust, I think about it like Slack, except I'm not slacking my colleagues, I'm slacking assistants, and they actually do this kind of work for me, and I can show them the kind of work. So it feels, Pat, to your point, something like a spreadsheet meets the ergonomics of a Slack as it, it's brought to me as opposed to I have to go to it. Uh, and that, that, that is, it took me a while to get there and now I see how the fragmentation is the power of what you're going after.

Well, and Gabriel, I have a quick question on sort of the psychographic of your user because your, your comment that it's like heat seeking for the people who are sort of ambitious and innovative and stuff like that. Um, I don't know if you have a name for them, but let's call them the makers, you know, the people who are not afraid to try to try new things and try to build stuff. Have you come up with a systematic way to find those people, or do they tend to find you through word of mouth or some other thing? Because that's not, you know, LinkedIn profiles don't say, you know, Gabriel Migger, right?

Like, I think it's a super interesting question at a couple of levels, but, um, our motion is, is dual, right? So the things that predict a great outcome with Dust, uh, I'm, I'm coming out of a core and trying Trying to think about what was most powerful about this call I had yesterday with the chief people and systems officer at the company that could not stop interrupting me 5 minutes into my pitch. He was like, yes, I did the talk on this. Yes, I've already read about this. I've got a blog post on this. Okay, when can I demo? Where do I put my credit card? Let's call you next week. And it's the top-down motion is enthusiasm and optimism about this technology changing most things for most people who spend most of their days in front of a computer. You need that. That's a necessary condition because I think it unlocks 3 things. One, it unlocks the belief in a horizontal platform for exploration, the ability for security to be in the support of business rather than a blocker. And genuinely sometimes example setting, like we have founders and leadership teams that are just like, how have you augmented your own workflows last week? And leadership meetings are being asked, they're doing offsites about like, how are you going to get better at answering some of your team's queries faster with us. So once you have that, then you have the right sandbox, I'd say, the right petri dish. I don't think we've fully cracked the builder identification. So right now it's more like bait. It's like the product is incredibly easy to use. Anybody can create an assistant even if they have not been labeled a builder by their organization. And it's just the sharing capabilities of their assistant that are somewhat throttled. But we can see from the way in which people explore the product, create assistants for themselves, share them with their teammates in a limited way, a great predictor of that type of personality. And if you ask me to look at LinkedIn and predict who are going to be in that family, I'd say the number one discriminator is somewhat to a degree, it's a bit ageist, but like people who are maybe earlier on in their careers who have a mix of tasks that they obviously know they can get an assistant to help with. And so they have use case one just laid out for them. People who have repetitive tasks and people who've scripted their way out of a lot of, of, of repetitive things before.

Just to be explicit, like we, we had the conversation. I think it's okay to say like it is people under 25. Like we were saying yesterday, the power users, the people that are using this all the time, um, at the companies are the people under 25 because they aren't set in their ways just to be explicit. And that doesn't mean everyone, you can be 70 and constantly innovating in a new way, but in general, they don't have the pattern that they've been set to. And by the way, that's true of a lot of the next generation of productivity. Notion, which Pat works really closely with, that is an under-25 power law type business. And, you know, the, the teammates here under 25 keep pushing me to transfer over to Notion. And it's just a different type of thinking. It feels like a very similar motion. At Dost?

Yeah, I think that the one thing we had, that we have, which is useful is that the immense B2C success of ChatGPT as a now obviously world-famous product has made it really easy to set up pilots by just telling teams, do you know what? Send a survey out, ask people how often they've used ChatGPT for personal use in the last 7 days, like rank by descending order, and that's your pilot team. That's the people you want to have poke holes at, kick tires, because they have, you know, we've asked the entire world to move from calculator technology, punch the same keys, you'll get the same result, to stochastic technology, ask the same question, you'll get a slightly different result. This has not happened. This is the biggest shift in, you know, the use of the tools that we have since the advent of the computer. We're asking an entire cohort of the workforce to move to a stochastic mindset. And the only way you get that is by having a risk-reward ratio that you're comfortable enough. It's like, you know what, I don't, I'm not asking it to be right 100% of the time. I'm asking it to give me a draft that saves me time many, many, many times over. And that distribution of ROI is something that I'm comfortable exploring with and iterating on. And I think that that is really one of the predictors that we see in people who've tried ChatGPT or in people who are just curious with new technology is they expect that some of it's gonna be a bit broken. But the upside scenario to them is so clear and so 10x that they're willing to make that trade-off or that local risk to, uh, to get things started.

So you guys have a lot of very strongly held beliefs, um, internally and externally. And the good news is you've consistently been right about the strongly held beliefs. You've named a few of 'em. I mean, you've talked about this shift from deterministic to stochastic way before it was mainstreamed. Um, you talked about rasterization and vectorization. I'll, I think about that. That can be unpacked if you'd like, certainly would need being unpacked, uh, on the show. Uh, if we go down that rabbit hole, you talked about no GPUs versus PMF, right? Can you just walk through some of the beliefs that Dust lives by? It can either be philosophical, um, as a couple of these are, or tactical. Like the Node GPUs before PMF?

Yeah, the first one is really the continued belief that focusing on products is the right thing to do because it really feels to me like we are only scratching the surface of what we can do with those models. Right now we are starting from the conversational interface, so that's why you use the Slack analogy. And I really truly believe that that analogy, the Slack analogy, will not sustain in time because the way we interact with that technology will change. It started with the conversation interface, but it will end in a very different place, in my opinion. Basically, those models are kind of the CPUs of the computer. The APIs and the tokens are really the bash interface. What we're doing right now is merely inventing bash scripts. And we have yet to invent the UI. We have yet to invent multiprocessing. We have yet to invent so many things. We are really at the the very beginning of what we can do from a product standpoint with that technology, whether it evolves or whether it stays like it is.

Yeah. One word that I think is going to be important, and I feel recent news has actually helped confirm or is an interesting new drop in the bucket for is the notion. So one of our product mottos is augmenting humans, not replacing them. And it's not just the naive version of saying like, we're not here to get people fired. It's really that we think there is tremendous upside in giving people who will still have a job in 5 to 10 years' time the best possible exoskeleton. And that it's a very different kind of company and kind of product conversation to be like, all right, how many dollars are we going to take away from your OpEx line next year? Versus this is the number of latent opportunities that you are not able to explore as a business because your people are dragged down in pushing like stale slideware around or not even knowing what dependencies they have on the rest of of the company. Like, this is how much friction you've imposed on the smart people you've spent so much money hiring because half of their day or part of their week is spent doing things that we should literally not be talking about in 2024. So that's one.

And the thing that comes back to, to, to the trope for a second, you've been saying that from the beginning, Gabriel. And in the beginning, you didn't use the word productivity. Like, you didn't want to use the word productivity. I don't, I wonder if that shifted. And if so, the nuance around why you chose not to.

I think productivity— there's two terms that I was hesitant on. Productivity to me sometimes feels like an optimization when really, um, there's two ways to be productive. There's doing the same things faster and there's doing just better things. And I think, you know, the mix effect of productivity is enshrined in effort versus impact. At the end of the day, your boss is never going to be mad if you spent no time doing the things you were assigned to do, but brought in the biggest deal for the company. Nobody's actually gonna make any comments on that being the bad decision because I think the more you grow in your career and the more you're close to the leadership of a company, the more you realize it's not about the effort, it's really about the impact. And the impact comes in sometimes unplanned, hyperplenary, completely like left-field ways where it's like, of course we needed to focus on this and it's clear in hindsight, but you need to free up time, space, energy, and mental cognitive space for that. The other one was enterprise search. I just feel like enterprise search is one that we didn't want to put on the website because retrieval of information is obviously a use case that people are very excited about very quickly. But we're just very convinced that looking for the document is a step that people are not particularly passionate about. Nobody wakes up in the morning and is like, I'm so happy that I'm going to get, just get the right document the first time around when I do the search. People just want to get their job done. and it just so happens that using context from 3 different documents across 7 data silos helped them get it done faster or better. I think the search bit is just, it's never the job to be done. Nobody really wants to search. They want to complete, they want to prove, they want to test, but the search bit is a step that we think will get abstracted. Going back to Stanislas Polu, I think that the interfaces and the experiences we have with this technology will really try to forget about what the original data source was. Quite fast, potentially, once we've gone over the trust hurdles that exist today. The thing that this all comes back to is collaboration, collaboration between human and non-human agents. And I think projects by Anthropic are an amazing example here. We thought about co-edition last summer. We have an amazing intern from MIT with us last summer. Who spend their time working on a co-edition interface. How do you chat to an assistant to make something that you're thinking about better, whether it's an app or a project or a document or a script? And this is something that obviously the recent release by Anthropic has made very palpable to many more people. That is to me the interface and the interaction that we need to get right. And that will be in the future. So we say augmentation, And we'll stick to it because I think it really helps us focus on the interfaces that help humans and non-humans make progress faster. It's going to be about proposals. How do I get to have a human in the loop with a proposal that's written just in the right way to decide if we swipe left or swipe right on it? It's going to be co-edition. How do I have the language of the human in front of the assistant be as easy to interpret and as foolproof as possible? For the final project to move into its final form as quickly as possible. And so you need that interface, that interaction between the agent and the human. And you forget that when you replace too quickly. When you focus on just replacing and removing, you've built something that is fire and forget, essentially. And you'll see the gains, you'll see the dollar gains. But I, you know, if you've automated 100% of your customer support tickets, You still need the insights from what people are pissed off about. You still need to understand and have your finger on the pulse of why people are stuck. Otherwise, you're slowing down your product development efforts. And product development efforts today live and die by some of the comments that are coming in from support tickets. And so how you've made that problem go away and become like actually maybe cheaper, sure, but also maybe virtual and harder to connect to is not, I think, a super long-term view. Of how your product and business is going to serve your customers best. Because you still need to think about the ultimate interfaces that are going to enable the decision-making to make it better and strategic and the best option for your customers in the future.

So keeping the human in the loop always. I mean, it is human loop is one way to say it, but it is drip, it's human driven. Like the whole point of all of this technology that we are building is to serve humans better. And as soon as you remove that, you've made it, you've made a terrible mistake because someone else is not going to do that and they're going to actually have a better experience with customers and employees and stakeholders. And then they're, they're going to win.

You know, obviously there's scenarios in which you're going to catch me and you're going to be like, you know, this one, this, you know, we know that humans get it wrong way more. And so we should obviously replace it. And this is a complex and nuanced problem. So I'm sure there's certain areas of it where pure replacement has fully understood, non-external, like with no negative externality value. But I'd venture that we're pretty poor at modeling where value is created and how it's funneled through the parts of our company today. And, you know, economists have been great at showing that when you don't price negative externalities well, we end up in pretty messy situations. And so this is, this is the question that I, that I pose to leaders who are asking, you know, what should I automate first? I'm like, well, I don't know, which parts of the company do you worry about the most? And often I just find that CEOs are panicked about what their customers say on support tickets. And so making that problem go away, making that problem less visible might be great for some OpEx conversations and your stock price could have unforeseen consequences if you haven't funneled it through in the right places. But also I think there's so much more to do than to shave 3% off your balance sheet. The spectrum of opportunity that you're giving your team, if this technology is in their hands and if they're able to come up with ideas, is broader than just firing people out of their job. And I'm not saying you shouldn't do that. Like, I think, I don't want us to be perceived as like naive in this ecosystem where the disruptive nature of this technology is going to take some people's jobs away because those jobs were currently being done by humans for lack of a better alternative. I think in certain situations you could see those jobs as having been created because we were waiting for the robots, having been framed in a way that was because we were waiting for the robots. But I don't know that that's what leaders of companies are excited by. I think that the upside, the future, the way in which we need to be resilient, antifragile for what's to come and what our competition is going to come up in. Those are the ways in which energy and support I feel should be fueled to, to support teams.

You guys, um, second-time founders, uh, you started your first company over 10 years ago. Uh, you were an early acquisition of Stripe. You guys were there super early on. What have you learned and done differently this time as second-time founders?

Um, I think, uh, really understanding that a few explosive bets are more likely to get you anywhere meaningful than over-optimizing too early on, on something that is still meaningless in the market. That's one thing that I think we think about differently. So like exploring versus exploiting, uh, um, and, and all those frameworks. Uh, that's one. Uh, I think the transparency that you could, the trust and empowerment that you give to your team is we, I don't think we were against it. It's more that we were clueless about how much more empowering you could be. So the idea, one of the best words from my Stripe years was paper trail. And it was, you know, you had two people in a corridor have a conversation and then one of them would take the time to just write a paper trail in Slack or in a document to say, you know what, we just had this exchange and we've moved the needle in this direction and it's saved and other humans the time and effort to go in a meeting room or figure out that this decision has been made. And, and, and it feeds a graph network of trust and respect for your coworkers that is, I think, second to none in how you can then just achieve more as a team. Um, so culturally you need to sort of push that to begin with because especially people who are earlier in their career will not always feel comfortable with how information should be shared. So I think that's one where example is important. Big markets that you really believe in for a long time. We loved technology when we started our first company like 12, 13 years ago. It was like, this is great. This is amazing. These are QR codes. Everybody's going to use them. And it's like, no, we have to wait for a pandemic to sell QR codes. Okay, I'll do that next time. And so like falling in love with the technology and not really fundamentally understanding how big the business could be if it's successful.

Successful.

And asking that question early and unabashedly is one thing that I feel is different.

So what we kept is our experience together. I think it's an unfair advantage to having built a company with a person because you've explored everything. You've explored the beauty, the terrible, the joy, the pain, and you know pretty much the entire API in and out. And so that makes, enables a much more efficient co-funding, I mean, co-founder interaction and collaboration. I think it's a really big unfair advantage. I think the biggest one that I think is completely different for me, and that Gabriel mentioned, is about empowering people. It's as a, really as a founder, it's not you, I mean, it's not to you early and it's to you to build and to build that initial spark, but then for the sake of the company, you are not the one that has to build. You're the one that has to make, create an environment for people to be empowered to build those things and explore and create new stuff. And the best value you can give is, I don't like to use that word, leadership was coming to mind. It's not necessarily leadership, it's really guidance and trying to create an environment where everybody has the chance to do what they want, but yes, in a guided environment so that everything works as a whole. But that, that would be the, the biggest difference and something that we learned about China at Stripe, at least personally.

So guys, let's move to a lightning round. We've got a couple questions for you.

All right.

Lightning round. Question number 1. Stan, you share these predictions for where the world of AI is going on Twitter from time to time. At this moment, what is your top contrarian prediction for where the world of AI is going? And don't give me this bimodal, a little bit of this, a little bit of that. Let's hear a point of view. What's your top contrarian prediction for where the world of AI is going?

I see it's the lightning, so I have to answer something. It's going to be tough. I think we're on the verge of entering a pretty tough period.

How so?

The excitement will go down. Maybe it'll take time to get to the next stage of the technology. There's tremendous value to create, but people will not see it yet. And it'll take a long time for it to diffuse through society. So there is massive amount of value to create, but it's going to be, we might have tough times in front of us.

All right. Short-term pessimist, long-term optimist. I'll take it.

All right.

Lightning round question number 2. And this is for both of you. Who do you admire most in the world of AI?

Ilya, He's just incredible. I've had the chance to work with him. He's my favorite people in AI. He's extremely smart, but he's not a genius builder. He's a genius leader. He's just a visionary. And I think that's been incredible. Karpathy, I know him. I actually don't know him, but I admire him a lot. And in terms of pure genius in AI, I think it's Shimon and Yaakov at OpenAI. They have crazy last names, so I'll let people look it up.

But Shimon and Yaakov are I'm impressed by those who've been around for a while and are good, you know, acting as good resistance and condensator elements in the system. They're just, you know, like providing the friction to remain optimistic, but cautiously so. And to me, one of the first, I think it was, I can't remember if it was a tweet or a podcast or an article, but hearing Jan de Kaeron to be like, You know, we can make pretty good decisions with a glass of water and a sandwich, and these things require power station sized data sources and are not making great decisions on some things. So we, we feel something is missing and it is like elegantly putting that back into, into, into perspective has been interesting to me because it's hard to not cave to the hype, I think. And so in some ways, pushing for a simple ideal like being open, which I think Yann LeCun is doing quite aggressively, despite that not always probably being the easiest decision. And also saying, you know, we probably haven't solved everything all the time is nice. And from my personal experience, the researchers that have worked for or with him have learned and taken from that quite a bit. And so that, and it's not French, but, you know, some touch of modesty, touch of temperance. I've appreciated in my discovery of the generative side of artificial intelligence, like after 10 years of just doing prediction and classification from fraud and risk and onboarding at Stripe and healthcare claims management and things like that. it's nice to, to feel like there's, uh, some people who've seen, seen a lot, done a lot, and are just questioning rather than, uh, affirming.

All right. So that brings me to the third and final lightning round question. You chose a Frenchman for your most admired, uh, Gabriel, and Dust is proudly made in France. Paris has been in an epicenter. Certainly an epicenter for all things AI. Your take on the Parisian ecosystem and what do you want to say for the French founders listening to this podcast? Other than I'm sorry it was in English, it's their fault, not ours.

Yeah, I think the French ecosystem is awesome because we, compared to where it was 12 years or 15 years ago when we started our first company, now we have talent because there's been a generation of scale-ups that went through the market and train all that talent. And most recently, that kind of explosion of AI talent as well, which is super exciting. So I think it creates a pool of talent and with the right conditions to create incredible companies. Obviously, it's not, I mean, tackling the US market from France is a challenge. And so that's something to be taken into account, of course.

Yeah, I think there's, uh, if, if you have ambition, there's, there's, there's a lot more to do. And then as long as you're not naive, where, you know, there are still some realities, like you, you can, you can fight some aspects of narratives. You can't fight gravity, or at least you shouldn't. You should probably work with gravity way more than you should fight it. Uh, but there's a ton more we can do. And I think we, we have to behave a little more like, um, tech countries like Israel, I think, in mixing ruthless ambition, a recognition for where talent is and how it's already connected and has high trust connective tissue, which I think is a great catalyst and accelerant in making great companies happen. But a recognition for where the markets are, where people are buying, where people are paying, and how quickly people are making decisions on shifting to new technologies, especially in that space.

And I think the biggest advice is, as a French funder, if you've always been in France, you have kind of that feeling that something magical must be happening in the US, something special. There must be something special about those people. Well, I'll tell you, I've been at Stripe, I've been at OpenAI, I'm working with Sequoia. These all are normal humans.

They don't have any magical capabilities.

They're just like us. And so it's really important to really be ambitious and believe strongly that you can, you can make it, you can do it, whatever it is, uh, from France versus the US.

Wonderful. That's a good place to end it. Thank you, gentlemen.

Thank you, guys.