Shortwave Rides the Tidal Wave: Inbox Agents, Hyper-Growth & Hiring AI Managers, with CEO Andrew Lee

Hello and welcome back to The Cognitive Revolution. Today, I'm excited to welcome back Andrew Lee, founder and CEO of Shortwave, for a conversation about the incredible speed of AI progress, how Shortwave is maximizing agent performance with today's frontier models and fundamentally reimagining digital communications, the ongoing transformation of the software industry at large, and company building for the AI era. The impetus for this episode was a beautifully exponential revenue growth curve that Andrew recently posted on Twitter. The sort that you can really only achieve with genuinely word-of-mouth-worthy practical value. Over the next 2 hours, Andrew takes us on a tour of everything that he and the Shortwave team have done over the last year to transform what was a useful but perhaps not quite transformative email assistant then to an email agent that is now routinely surprising and delighting both users and Andrew himself with the increasingly complicated projects it can tackle. So much so that Shortwave is now expanding beyond email and reconceiving the product as an AI agent to help manage communication across all major channels. Having concluded that AI makes software so much easier and faster to create, such that speed is really the only moat going forward, Andrew does not hold back on lessons learned, and the technical insights here are outstanding. Andrew breaks down how they've completely rebuilt their infrastructure at every level. By constantly testing and swapping in new models, moving to Pinecone's serverless offering for their vector database, and adopting a hybrid structured plus vector search paradigm that delivers better results at lower cost. Perhaps most fascinating is Andrew's perspective on agent architecture. While many companies are pursuing multi-agent approaches with specialized subagents, Shortwave has found better results with a simpler approach that makes careful use of Anthropic's caching features to support long-running tasks with lots of context while also maintaining positive margin unit economics for the business, but otherwise largely trusts Claude to act effectively as an agent, both by calling the right tools and determining for itself when it's found what it's looking for. Personally, I've had a number of wow moments as a user. I was honestly a little nervous to allow it to organize my inbox for the first time, but now I'm making regular use of the conceptual to-do lists that it's created for me. And it also saved me a cool 30 minutes the other day when it collected all receipts from a recent trip and compiled them into a tidy expense report. In the last third of the conversation, we turned from the product itself to the question of how to structure a company for success in the AI era. Having recently closed another round of venture capital, Shortwave is hiring for a number of roles, which Andrew describes not as traditional individual contributors, but as AI agent managers. across software development, marketing content creation, and more. He plans to keep the team quite small, targeting just 15 or so employees for the foreseeable future and prioritizing talent density and speed of execution above all else. With that in mind, he's offering a $10,000 referral bonus, including to listeners of this podcast. Finally, before getting started, I wanna note that this episode is brought to you ad-free by Shortwave. I've mentioned in the past that we are experimenting with sponsored episodes, that allow companies with a timely story to cut to the front of the line. Of course, my commitment to you, the audience, is that our bar for interesting content and my preparation process will remain the same as always. Andrew and Shortwave were really a perfect fit for this opportunity. Their product is clicking, their business is booming, and he was eager to get his hiring message out sooner rather than later. As always, if you're finding value in the show, please take a moment to share it with friends or colleagues who might be interested. Leave us a review on Apple Podcasts or Spotify, and I always welcome your feedback either via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. Now let's dive into this fascinating conversation with Andrew Lee about Shortwave's AI-powered transformation, not just of email, but now all digital communications.

Andrew Lee, founder and CEO at Shortwave. Welcome back to the Cognitive Revolution.

Thanks for having me. It's good to see you again.

Yeah, it's been, boy, a lot gets packed into just a year in the AI space. And I was looking back, it's been just about a year since your first appearance on the pod. And indeed, a lot has changed. You know, at the time I called Shortwave the AI email assistant that I had been waiting for. And then as we were kind of catching up in preparation for a second conversation, you said, At that time, it only kind of worked. And I was like, yeah, I guess that's true. I look back at these things that seem so mind-blowing to me at the time, and obviously we've way surpassed them. But what caught my eye and got me to reach out again was you had posted a graph of shortwave revenue on Twitter, and it's basically the canonical exponential curve where it looks like it's on the verge of going totally vertical. So to kick things off today, what's new and, you know, what is working now that is leading to such incredible growth that was only maybe sort of working a year ago?

Yeah, it's, it's really been an evolution of that AI assistant that, you know, we talked about last year. I think the thing that you played with a year ago, uh, you could chat with it, it could answer your questions. It did an okay job of searching. It did an okay job of writing your email, but It wasn't really that smart and it wasn't really that trustworthy. And if you asked a question, you know, like, hey, where's the receipt for this? Like maybe you'd find it, but you couldn't really trust that it found it. And it also couldn't do a lot of the normal stuff that you want to do in an email client. Like if you go in there and you say, you know, hey, what are the most important emails I have? Or, you know, archive all the cold sales emails or something that I got. Like it couldn't do any of that stuff. Couldn't manage your to-dos. It didn't have no idea who your contacts were, your labels were. It was just sort of like a cool search and a writing thing and like maybe like a general purpose AI thing, but it wasn't quite like having an employee sitting next to you, which is the pitch they were trying to make. And we have iterated our way to a thing that actually really delivers on that. Like you use the thing and it kind of does what like a virtual assistant would do for you. And it actually works and it can do almost everything that you as a human can do. So. Yeah, I think it's just sort of reached a tipping point where people are like, holy crap, I can do my email not by like doing my email, but by talking to a thing that then does my email for me. And that lets me think about things at like a much higher level and be like way more productive.

Yeah, I've experienced quite a bit of that and can definitely testify that there are some pretty wow moments that I've experienced. I mean, we, I don't know if you wanna start with use cases or, or sort of how it works under the hood. Well, maybe start with use cases. One that I tried, which I thought was really interesting, was to just ask it, take a look at my last 100 emails sent and give me whatever advice you have. And, you know, you learn there's a lot, you got, you got a lot in the outbox that says a lot about you. And I honestly thought it gave me like pretty interesting advice, which was both apt in some ways and then also made me think like, maybe I'm not exactly spending my time 100% the way I aspire to be spending it, just because of the sort of balance of things that it was seeing. What are some of the other exciting use cases that you have got the most value from or that maybe your customers have surprised you with?

Yeah, email is a crazy valuable corpus of stuff about everything involved in your business or your personal life. So obviously it has all your human correspondence, but it's got All your SaaS notifications and it's got all of the attachments, you know, all, all the files and PDFs and things that come along with that and all your calendar invites. And like, we just know a ton of stuff about you. And I think if you had asked a human like, hey, go through my email and gimme some advice, and they were a reasonably smart human and they had the time to do that, they would also give you some good insights. You just, you never do that because it's a lot of work. I'd like, but the information's there. I think the prompt you sent me, like you said that the other day, like I think that's a super fun one.

It's super cool.

I actually just this morning had a prompt where, you know, just yesterday we rolled out a big UI change. And anytime we make big changes in our product, it's controversial and we get a lot of feedback and it's always scary. And I wanted to know, hey, how are people responding to this? And we have this new sharing feature that lets us share like all the support threads that come in, get shared across the whole team and get available to the AI. And I asked the AI, how many people have emailed us and complained about this new layout in the last 24 hours. And I got a report. It was 19. There were 19 users who reported it, and it gave me a summary of here are the top 5 reasons. And it was super useful and super good. And for me to consolidate that myself would have taken me quite a bit of time. And I got a snapshot right away of like, okay, what's the reaction? How are people feeling about this? What are the top things we maybe need to address? So that was a big use case for me personally. We've seen a lot of other fun ones in the wild. I think one of the most common ones is People sort of start their day with a complicated prompt. So, you know, you're in sales and you have 8 customer demos coming up and you have an inbox full of emails from folks and you're just like, hey, help me figure out like, what are the tasks I need to do? What order do I need to think about things in? What should I remember for each sales call? And they'll have this big prompt. A lot of times users will share these prompts with us of like, this is the thing I use to start my day. So that's a pretty common one. I think another really common one is people doing attachment analysis along with their email. So there's a lot of like real estate agents and general contractors and architects and stuff like that where they email back and forth all these PDFs all the time. And they just need to answer one question from the PDF. Like, you know, what was the specific payment terms on this contract or something? And they can just ask the assistant and it can like read the whole PDF for them and give them the one answer they need to like write the email. There was another example that got shared on LinkedIn the other day with one of our users who was selling his house. And he needed an inventory of like all the furniture that was in his house. And all that stuff was in his email somewhere, right? Either in like emails with his wife or receipts that were sent or whatever. And he was able to get a full inventory of like all the furniture they had purchased over the years for their house in a nice consolidated, accurate report with just like a few keystrokes. So lots of interesting use cases. They're all sort of big prompts, but spanning like a whole wide spectrum of things.

I did a small version of that inventory thing after a recent trip where I had to collect and submit receipts for an expense report. And that was another— I mean, it's a very mundane task, right? But in a way, it's like the perfect job for AI to handle these things that I otherwise kind of dread. And it was cool to be like, find me all the receipts from my trip. There should be Ubers, Lyfts, and a couple DoorDashes in there, and just have the whole thing pop out with values and everything sort of itemised. And I was like, man, it's another one of these moments where I was like, this AI thing could really catch on. I mean, I'm sure there's more, actually. I do think that one big barrier to practical value in AI is just kind of lack of imagination sometimes on users' part. I certainly feel like I'm guilty of that all too often. When I see something else somebody has done, I'm like, why didn't I think of that sooner? Any others that you want to share at the top here that are just like things that should get people's wheels turning about how they might, if they're just a little bit more intentional or creative, get more value?

I'll share one of the most creative ones that I ran into. So this was another one that a user sent to me. And the thing is, I personally am still figuring out how to use the thing. And often the insights are like, user sends me a thing of like, why doesn't this work? Or check out how this thing works. And I'm like, I never thought of that. So here's what I never thought of. There was a user who was using us in combination with another SaaS tool. I think it was Linear. And they wanted to be able to like extract action items from their inbox and like add them as tasks to Linear. And we don't have a Linear integration. Maybe we should, but we don't. And it turns out that the LLMs actually know the structure of the URLs needed to create tasks in these other SaaS products. And so if you go and ask the LLM, create a link that when I click it will go and do this task in this other product.

Mm-hmm.

It actually works. And so he had a bunch of custom prompts where he's like, you know, extract action items from this email thread, you know, and then give me some links that I can click and each one should create a thing. And like, here's my base URL for my project and this other thing. And then the LLM spits out a bunch of links. He clicks all the links, boom, he has tasks in this other product. And you can build integrations with other things without us doing anything and without any code being written. It's just a prompt. And the LLM's knowledge of how to construct URLs.

Yeah, that's really very creative. I like it. And also kind of an interesting window into just a future in which AIs are potentially more and more likely going to be sort of solving problems in unexpected and maybe at some point even hard to interpret ways. Although that one's pretty simple to interpret. But the creativity there is on both the user and the models part is pretty impressive.

I'll give you one other one that I thought was pretty fun. There was a user wanted, they wanted to do a mail merge and, but they wanted to have a mail merge where like every email really looked custom, where like the AI was like going off and like searching their email history and like finding an interesting fact about them. And like, they really wanted that user to feel like they, you know, they were paying attention to them. And so they took a text file, they put a whole bunch of email addresses in the text file. They uploaded that text file into the generative AI system and they said like, loop through the emails in this text file. For each email, go and search, find emails that I've sent with this person, figure out like a nice, you know, greeting for this person, write an email. And so they fed in the text file, had it loop through, it wrote a very custom email for every single person. Each one they got to review it, click approve, click send, and they could send, you know, 20 emails that all felt very custom in like a really organized, fast way with the AI.

Yeah, that's cool. I was actually gonna ask you about supporting loops, but I hadn't tried it myself, so it's cool that that is already something you're seeing.

It works.

In the wild.

Yeah.

So what have been the, you know, last time we got like quite deep into the guts of how it worked. You have a background as a database whisperer and, you know, there was a lot of talk about kind of how the index happens and you're, you know, you take everybody, every email out of somebody's inbox and store it in your own system and have your own indexing sort of thing. And then the model can kind of build on top of that. If we work from the ground up, how much change has there been at that foundational database layer and sort of retrieval layer versus how much has come at the higher layers of the model and then the sort of patterns of behavior that you're getting the models to actually exhibit? Yeah.

I, so honestly, between the time I was on the pod last year and now, I think basically every part of that stack has been completely rewritten. You know, we are using a different embedding model. We are using a different vector database. The search stack has been completely rewritten. The API to that search stack has been completely changed. The models we're using for the agent are different. The agent code has been completely rewritten. So yeah, top to bottom. And I think driven by changes in the capabilities of both the model and also our understanding of how to apply those models. And every time the model can do X better, we're like, actually, if we rewrote the system like this, we'd get this extra unlock. And so the thing is just evolving at a truly phenomenal pace.

Yeah, that's undeniable. And although some do try to deny it, maybe let's go a little bit deeper on each of those levels. I'd love to just hear kind of what you've learned and maybe also to the degree that you think it's applicable for other AI builders, what sort of upshot or advice comes out of some of these different changes at the database layer? Do you have a favorite vector database that you would recommend to others at this point, or what have you learned that caused a change there?

Yeah, so we've made a bunch of changes here, and as a little bit of context, one of the big unlocks for us here was having a model and an agent on the front end of our app that was able to reason a lot better about how to use search. And most importantly, was able to reason about how to run multiple searches. And this sort of simplified the requirements for us on the search stack, because it used to be the model would run one search and it would try to, it would give you, you know, like a semantic component in the search and you try to run that search and you in the search stack had to find the right email in that one search because you get one shot at this. And now that we have an agent that can run multiple searches in parallel or also can run them in sequence, so it can try a search, it doesn't find it, it can try something else, or it can like try a search for one thing, find some information, and then adapt and try another search. It's actually allowed us to simplify the backend implementation and kind of focus it on a much more narrow task and then be much, much better at that. So the evolution of our search stack in the backend is driven by, in some ways, the simplification of requirements. And we, we focused on how do we make it really good and really fast and really reliable and really cheap to do this kind of more narrow focus tasks. So we use the Pinecone serverless offering as our database. We used, we used to use their, their pods, but the serverless offering is a much more cost-effective solution because it separates storage and compute. And so you can kind of tune for your use case. And we use a lot of storage because you have a lot of email. We use an embedding model now called BGE, which— and we use a bigger embedding model now than we used to because the serverless offering kind of unlocked the ability for us to just use these bigger vectors without burning too much money. And I think one of the big changes is we started to use hybrid search on the backend. We used to use this pretty complicated pipeline that combined some like smaller model LLM calls with some like very particular types of like feature extraction and search. And if you look at— listen to the podcast last time, you can remember how we did all this. And there was like a re-ranking step that we did. The whole thing was like very slow and complicated and very brittle because there was a lot of sort of custom stuff in there. And what we've done instead is we've allowed the Search API to specify a semantic component and some constraints. And the constraints are sort of like normal email query constraints. You can say, hey, I want only emails that are like in this date range or like with this contact or with this label and the semantic component, we run like an embedding search with Pinecone. We also run a keyword search and there's an algorithm to sort of combine the keyword search, like the full text search results with the semantic component and like get a score, kind of rank based on that score. And then we filter based on these other components. And so you get this API where you can say, hey, I want emails about this topic. With these constraints, and we can very quickly give you a very accurate list of those in a way that's like cost effective to scale across everybody's email. I think this has been huge for us. It's a lot cheaper to run. It's a lot faster. It's a lot more reliable and is also producing just way better results when you combine that with an agent that can reason about like running multiple different queries.

There's multiple things there that I think are quite interesting. First of all, just the fact that the whole stack kind of has to obviously work together, but also an improvement in one layer of the stack allows for simplification in another layer of the stack. That's definitely been a huge theme for me over time with the work at Waymark too, the shenanigans and hoops that we had to jump through to choose an image for a user once upon a time. were just like comical almost in their complexity, and also that didn't work that well. And in a similar way, now it's just like feed them into a model and it usually picks pretty well. And it's like, we've cleared out so much old cruft that we developed to kind of get that first version working. And now it's just like a simpler and better solution driven by model progress. Did I understand correctly, though, that when you do this search, is it running the full Like the filter comes after that. That's kind of interesting that you're doing like the full vector search through the entire corpus, if I understood correctly, and then filtering only after getting results back.

It's a bit more complicated than that. I'll give you the somewhat simplified version, which is essentially that we run 2 types of searches. One of them is a full-text search that is constrained based on like the keyword and the metadata components of the filter. The other one is a semantic search that we run in Pinecone, and we combine the results of those 2 queries and then we score between them, and that gets post-filtered. And so it means there are situations where if you had a lot of good semantic results for something You could potentially miss the best ones, but I think in the vast majority of like real use cases, the way that we are, you know, combining that full-text metadata constrained search and the semantic portion and post-filtering is generally finding the best results. The true we are actually going to score every email across everything and apply these. these filters. If you really wanted to have that be 100% accurate all the time, that would be sort of an intractable problem. But we can approximate it very closely with our solution.

Yeah, interesting.

Okay.

It seems like model progress has driven a lot of value for you, but also the scaffolding and actually shaping the behaviour of the thing is also super important. One thing I've definitely seen repeatedly in my usage Both when I accept the invitation for the AI to organize my inbox and also in some kind of random idiosyncratic things that I've done. Like this one really, it was like, man, the AI and I are, are like really on the same wavelength. I have the total inability for whatever reason to remember which of my contacts is the stronger and which is the weaker prescription. And every time I go to change my contacts, I have to search in my email for this particular thread where I know that information is contained. And so I'm always doing this search, and every time I've always been searching the traditional way, I end up running multiple searches. I'm like, okay, I think the word prescription was in there, and then there was— I know the word corrected is in there, but what is it? Right? And so eventually it takes me usually like 2 to 3 searches to get there. So I asked the Shortwave AI assistant to do that. And it was interesting to see it basically go through the same process that I went through, where it was like, search, get some results. I'm not catching the thing that I want to catch, and then search again. And I think it took like 3 or 4 rounds before it finally landed on the thing and answered my question. More often, I think people will engage in probably higher level things. The organise your inbox is a great example where it sort of just kind of goes off and, I mean, it comes back and gives you suggestions and opportunity to confirm and make the movements that it suggests that you move. But when you do accept it, it just keeps working. So maybe take us through how you've thought about creating the agent behavior. What has been the big unlocks at the model level that have made this possible? And then what have you had to do to wrangle it into something that is actually valuable for people on a day-to-day basis.

Yeah, and this has been the biggest shift in our product since we last talked. I mentioned, like, every part of the stack has been rebuilt, but the basic functioning of the assistant has gone from a single LLM call that produces the final output and sort of a complicated Rube Goldberg machine to, like, get the prompt right for that one thing. And we had all these complicated heuristics and rules and whatever to do this, and, like, smaller LLM calls. And we've thrown all that out and said, what if we just run the big LLM a whole bunch of times repeatedly until we get the right answer? And we tried this. I'll go through the history a little bit. So a while back, let's see, like 2 Decembers ago, I think OpenAI rolled out their tool calling features in GPT-4, and we tried them and we didn't think they worked. Right. Our general experience was As soon as you try to get it to call a tool, like the whole model stopped reasoning well and just gave you bad answers. And it was much better to just like tell it, you know, format something in XML and we'll go and do the tool call for you. And even getting to do multiple tool calls, it didn't really work very well. And so we kind of kept this sort of rules-based system. And then last summer we tried this again with GPT-4o and it kind of worked. And we said, hey, well, what if instead of this kind of one-shot approach, we had a multiple-shot approach? And originally it was very much geared around search of like, well, The search use case of running multiple searches seemed important. What if we ran multiple searches? And so we kind of rewrote it a bit to do that. And it worked better, but not like dramatically better. And we have a launch actually in September that was like, hey, our, you know, our, I think we've called it our V2 agent. Like, here's that new agent. And it definitely got us some growth and some excitement, but it wasn't a major thing. And then in October, or maybe, maybe November, I was listening to a podcast from the founders of Bolt.new. they were talking about how they built their stuff. And my memory from that podcast is basically they said, hey, you got to try Claude Sonnet 3.5, the October version of it. It's different. And by the way, Obold.new is open source, at least parts of it are, and you can kind of see what their prompt is and how it works. And I found that very interesting. And so I did. I tried Claude Sonnet 3.5 specifically for tool calling, the October version of it. And it was dramatically better and it was able to like GPT-4o, you know, you could, you could have it, you know, you call it, it spits out a tool response, you feed the tool back in, you call it again, you know, it spits out another tool response. Like, you could have it kind of iterate a few times, but it would sort of like go off the rails after, after too long. So we didn't want to do too much of that. But with Claude 3.5, it could go on for a long time, right? It could run many searches and do many things and kind of still stay on track and still keep reasoning. And it seemed totally different. And so we're like, okay, maybe we should rewrite this whole thing and say, we're going to have this new approach. We're just going to call plots on it over and over and over again. We're going to let it run not 2 or 3 or 5 times, but like 20 times. And we're going to put all of our smarts into like really good tools and a really good prompt for the overall thing and like a nice agent framework around that. And we're just going to let the model reason about what data it wants to pull in. And this was a total rewrite. Like, we, we had a lot of, like, very custom email-centric stuff, and we said, no, no, we're going to have, like, a fairly generic agent framework at the core. We're going to build a whole bunch of really nice tools around it. We're going to have a lot more tools than we used to have, and then we're going to iterate our way to a solution. And this is the agent that we launched in January. And if you look at our— you mentioned the hockey stick growth graph, like that little kink, that's the V3 agent where we rolled this thing out, and it was just a whole lot smarter. It's just It could start solving like very general, very open-ended things. And there were a lot of places where the old version didn't work that iteration solved. I think that's the core. When people are talking about agents, I think what they're really talking about, if they actually have a working solution, is iteration. It is, it tries a thing. If it works, great. If it doesn't work, it tries another thing. And sometimes that's running 3 searches in a row. Sometimes that's like, you know, trying to run a search. Finding out that the search, you know, criteria that you specified is actually like malformed, having our system throw an error, having the LLM see the error response and try again. Or sometimes it's like trying to schedule a calendar invite, having— realizing that the calendar invite is like conflicting in some way, having our system spit back some information being like, this is conflicting, and then trying again. So you get this feedback mechanism where over a series of like many LLM calls, you can iterate your way to an answer that no single LLM call could have produced and get really good results that way.

Okay, I'm going to dig in a little bit more. We have time. It would be a very natural time to talk about sort of the expanded view of the company and the product. But before we get to that, one thing that I've just been kind of wondering myself is, like, How does a model— this is one of the challenges with RAG apps, whatever. And a lot of people have built RAG apps and have come to some frustration with them, where I think often the core problem is, as you said, you do one search and then the model just kind of has to do the best with what it's given. And if the right information isn't in there, then it's really going to have a hard time definitionally. Expanding to allowing for iterative search, definitely I have felt makes a qualitative difference in terms of how likely it is to be able to pull the thing back. I don't think it would have found my contact prescription a year ago, but I still wonder, how do you think about maximising that, given that the models don't have the thing that I have, which is I know it's in there and I know when I've found it. And this seems like a sort of fundamental challenge from the model's perspective, that how do I sort of know when to call it? When do I decide that this amount of information or this particular information is, in some sense, satisfying or the best I'm going to get versus feeling like the way I feel, which is like, I know that that's not quite it yet, but I know it's in there, so I'm going to kind of keep looking. And then when I do find it, I'm like, it's always very clear to me, like, yes, this is what I was looking for, and I know it with high confidence. So the models don't have that. How have you approached the problem of leading them? I can't imagine that they're entirely just doing it on their own. So how do you help them decide when they have found enough versus when they need to keep digging?

So I'm going to give you kind of a complicated answer here. So the first thing I'll say is, The models actually do have some of that because they may not know, you know, what you know specifically, but they know, you know, what it looks like generally to have found the thing you're looking for. And they, they have a lot of training on just emails in general and what emails look like, what people expect from emails. And so we found, so for example, we have this organize your inbox feature and it goes through and it finds what we described as like low quality emails and it gets rid of them. And I have been shocked at how little work that we have to do to describe what a low-quality email is. And so for example, like it's really good at spotting cold sales emails, even though like you might think like, hey, how do I know it's a cold sales email? Maybe this is somebody I know, but there's something about the tone and style and whatever cold sales emails that like really stands out to them and then they spot. And I found that to be like with there, we do have some prompting work to make this really clear, but like we didn't have to do a lot to tell it this. So that's my first thought. My second thought is we do try to give it tools so that it can go look up this information itself. So if you're like, hey, you know, I'm looking for an email from, you know, some important investor or whatever, it maybe doesn't remember who your important investors are, but we have a contacts tool. And the contacts tool, when it returns results, we put in there using statistical information how important this contact is. So it can actually look up who is important if it wants to. It can also just go search your email history and find like, you know, emails you sent and make assumptions about, hey, if you've recently exchanged emails with this person, it's probably important. And so this is an area we're trying to get into more and more of, of if you ask it some open-ended question, kind of use the tools it already has to reason about in some reasonable way. Look at your email history, look at your contacts, look at your calendar events. Have you met with this person recently? Those types of things. And try to, try to get some of that context. The third thing I'll note is I see a lot of room for improvement here. We today don't do anything to capture sort of the triage actions that you take. So when you use Organize My Inbox in the morning and, you know, some of the access— the suggestions we accept and some of them you don't, what we really should be doing is remembering all of the things you did and then sort of customizing it to you over time. And today we don't do that. If you want us to change the way we do triage for you, you can actually tell the AI, you can say, remember, you know, never archive newsletters from this sender. And like, we'll remember that. But you have to be explicit. We don't remember just based on the actions you take. So I do see a lot of opportunity there.

Yeah, that's quite interesting. I just did an episode with the chief scientist, Guy Gerari, at Augment. And yeah, I mean, there are multiple similarities, actually. I mean, they also have a big emphasis on just ingesting a ton of code right off the bat and putting it into this specialised index, but also the behavioural part. They've developed a process they call reinforcement learning from developer behaviours. And it's about kind of both observing what the developers are doing, but then also how the developers are reacting to what the AI is bringing to them. And it sounds like it's working quite well for them. It seems like one big revelation, reinforcement learning works. So I see that probably coming quite soon in your future too. On just another kind of random question, but an interesting one to me at least. Claude obviously is significantly more expensive than some other options. So what I see typically from the agent, I don't know if this is like super consistent or just kind of my observation, but typically it seems like it's 10 threads found. And then it kind of does its sort of reasoning and do I need to search again, whatever, based on the 10 things that it's found. The other way I could imagine doing that would be maybe using a slightly lesser but way more affordable language model to do the evaluation on stuff that came back. So if you used Gemini Flash, for example, I think you would be able to handle like 30 times as much And you could maybe complicate this with caching. I'm not exactly sure how you're using some of those more advanced optimizations, but first order approximation, you could handle like an order of magnitude more search results if you put it all through Flash to assess relevance versus saying to Claude, like, you handle this all yourself. Are you doing any sort of that kind of ensembling or using different language models for what they seem to sort of specialize in, all things considered? Or is it just all Claude all the time because it really is just that good?

So we do use a bunch of different models, but we use them for different features. So for example, our autocomplete is actually using a fine-tuned GPT-4o Mini. Our quick reply suggestions, like the one-button reply suggestions that we give you, those are actually using Llama 3.2, the 3B model. We use GPT-4o, just like not Mini, but GPT-4o in a few places. So we do use different models in different places for different things. We have looked at sort of stringing them together, you know, like starting to save costs for certain things in the assistant by outsourcing other models. And I think our experience has been that it introduces a lot of complexity and it affects the ability of the model to kind of reason across lots of different types of activities in a sane way. If you just feed all of the data into one big model, it can think about, you know, lots of complicated relations between different things and it can come up with ways to use tools and ways you didn't think. And it's been a big unlock. for us, and the cost savings haven't seemed worth the complexity and sort of the loss of generality of, of like having this turn into more of a pipeline between different types of models. The other thing I want to note is the caching that Anthropic has is really, really critical for us. It's a really big deal. So when you have this agentic flow, you call this, you know, the same history over and over and over again. You only append to the bottom, and our histories get very long. Like, we can have hundreds of thousands of tokens at any one of these calls. And the Anthropic caching is a little hard to use. You have to really construct your agent to be very careful about keeping like earlier things immutable. And we've done a lot of work to make that happen. But if you get it working, it can save you 90% of your cost. It's a hugely impactful thing. And frankly, if we didn't have that, we could not afford to run— like, we would be just at Anthropic cost, we'd be losing money every user by a huge margin. So that's been a huge unlock for us. It also is One of the reasons we switched to Claude actually is even if the models from OpenAI could use tools as well as what we have for Anthropic, we couldn't afford it because the caching from OpenAI is just a lot less of cost savings. It's only like a half off.

Yeah, half off and pretty much automatic. You don't have to do anything, right? Versus, can you tell a little bit more about how— I know Anthropic is like 90% off once something is cached, but there is also a like a one-time cost to get something into the cache, right?

Yeah, but it's not an additional cost, right? So you, at the time that you run it, you have to tell it to cache it. And I think there is maybe some slight additional, but it's not a significant additional cost. And then every iteration after that is 90% cheaper. And if the average, if the most common action is like people organizing their inbox, which it is, and that is 20 tool calls, it adds up really quickly. Um, I was actually, I was actually just at OpenAI yesterday and I was just talking to the agent SDK folks about this and about caching. And this was frankly the number one question from the room, from the other founders I was with there was like, what about caching? How do we get caching to work? Because I think everyone's figuring out that you're going to be calling these models with the same context over and over and over again and making that work efficiently really matters a ton. And The OpenAI caching, you're right, it is a very simple API, right? It sort of automatically figures out, you automatically get cost savings, which sounds great. And the Anthropic one's much harder to use, but the magnitude of those savings is quite different.

So on the Anthropic side, is it smooth to the point where you can sort of build a cache? I've used the caching a little bit, but it sounds like modular, some possible implementation difficulties. You can extend and also extend the cache iteratively as you go. And basically it has all the features you really need to realise the full savings in the sort of sticker price of 90% off.

Yeah, you can. So you sort of checkpoint at every iteration in your agent flow saying, okay, I cache up into this point, and you have to construct your agent to make use of that well, but it's totally doable.

Interesting.

So are they actually caching? I wonder if there's a big optimization there for them under the hood where they're like, it sounds like they may be caching multiple versions of each conversation.

I don't know. I actually am very curious what they're doing. Maybe they're just eating the cost. I don't know. But our bill goes way down, so it's great.

Yeah, that's interesting. How do you feel about this sort of A huge question in the AI space generally to me is, are all the frontier providers converging or are they diverging? We've got somewhat different caching things, but maybe OpenAI is going to get the message and sort of do a more Anthropic-like one. We've also just seen in the last day or two that OpenAI is going to embrace the model context protocol that Anthropic led the way on. Do you feel like these things are converging, diverging, some complicated mix of those?

I think it's some complicated mix of those. So it has been nice that they've been, I think, converging in terms of the API surface. So we now very easily can swap between different providers and all the tool calling and stuff lines up perfectly and makes it really easy to do that. So I think that's been a very nice thing. And Everyone is understanding that like there should be caching and are hopefully converging on some of the ideas of how to do that. MTP is maybe becoming a standard that everyone's respecting. So I think there's been standardization in the interfaces. It does seem like the different labs are focusing on different things. So it seems to me like Anthropic is really focusing on this iterative approach. It seems to me that OpenAI is maybe caring a lot more about reasoning and potentially more about like multimodal. And you'll notice that we use, you know, many different models. We use models from 3 different vendors. And a big part of that is we feel like each vendor is bringing something different to the table. So one example where I think OpenAI totally crushes it is their serving infrastructure is super performant and reliable relative, like Anthropic's APIs, they go down a fair amount and time to first byte is slower and stuff like that. But so for example, for our autocomplete where latency is really critical, we use the GPT-4o Mini with a fine tune and like the serving stack from OpenAI is like super good. So I think that's a differentiator there. We use, uh, for the summaries and for the instant reply, like quick actions and for a lot of the stuff that where it's like cost and latency matter a ton, but we don't need a lot of intelligence, we use open source models. So we use the Llama 3.2, and that's running just like on Vertex on GCP. So it's great to have, as someone building an app, to have many different vendors converging on standards, but focusing in different areas ends up being really nice for us.

What else have you learned about agents in this journey? I mean, everybody's setting out to build one. I think you are pretty far ahead of most. Have there been false starts, the things that you sort of expected to work that didn't, or just Any kind of unobvious lessons learned that you would share with folks who want to build their own agents?

I think that the biggest thing I would say if they're thinking about it now is that things are different, um, in just in the last few months, right? Basically, like if you tried this before October of last year, or if you've never tried the Anthropic models, like it's different now. The stuff that you tried before, it actually works. The cost actually with the, with the caching stuff is manageable and you should try it again. And I think you'll be really surprised. And this is, yeah, you'll notice stuff like Bolt.new has gone totally crazy. And the new Cursor agent mode, I think a lot of people are still using Cursor for the autocomplete. Like, that's old school. Use the agent mode. It's totally, it's totally insane. So, yeah, that's the big message is it works now. It's different. You gotta try it out. It's gonna be, it's gonna be amazing. I think one of the big things we've learned is people have no idea what the UX should be around this stuff. And we're very much still figuring this out, but You know, people kind of figured out like, oh, autocomplete, it works like this. And like people, you know, everyone kind of wrapped their heads around how this sort of thing should work. You tab, it's complete. And now we have a thing where it's like, it's gonna go off and just do a bunch of work for you. How do you do that in a way where the user doesn't get really uncomfortable about, is it sending emails for me? Is it changing my code in ways I don't realize? And we're gonna need some UX for like sort of oversight and approval and providing guardrails on these things that like doesn't get in the way. But still gives you confidence that it's doing the right thing. So I think that's going to be a big area for us. There's just a lot of questions, a lot of unknowns given how new this stuff is, but I think it's going to be really exciting to see how it unfolds. Yeah.

One product feature that I honestly haven't used as much as I maybe should, and kind of because I'm not sure quite what it's going to do honestly, is the AI filters. And I was both curious to know like how that works under the hood? Is it sending all— because what I think is going to happen is I'm going to say, filter out this kind of email, and then emails are going to come in. And I guess then literally every one would get sent to a model to say, is this the kind of email to which this filter should apply? And I've only been a little— I mean, I like the sound of that because I got a lot of crap that I need to filter out. And historically, I've definitely spent way too much time clicking on those myself. But the flip side of that, of course, is like, if I create a workflow where I don't have visibility into what's getting filtered out, I have been burned by that in the past as well, even with just like Gmail priority inbox over time. So first of all, am I intuiting correctly what's going to happen there? And I guess more broadly, how are you thinking about these sort of next level agentic scenarios where We're going today, the AI system brings me stuff, it's up to me to confirm. But it's not hard to, with all the talk of how well it's working, it's not hard to extrapolate a little bit and imagine that, okay, fine, just go ahead and do it. So how are you thinking about those next moves into actually taking actions without necessarily a human in the loop approving each one?

I think you've caught an interesting Difference in that feature, which is it is the only AI feature that we have where the AI just does things and it does. There's no approval flow from you, and it's actually a really popular feature. So the basic idea is like you could write a prompt and you can choose three actions based on that prompt. You can either archive it, you can apply a label, or you can delete it. And we've had this live for a couple of months now. People love it. There hasn't been a ton of problems in terms of people losing stuff, but we do get support requests sometimes where people are like, "Hey, you're missing my email," and it's like, "Well, let's check your AI filters," and then they have one set up and. So I think there's no way of getting around that entirely. But the dream here is that becomes like a run of the full agent. Right now it's a very simple implementation. We do use OpenAI for this. This is a GPT-4o mini task, which means, yeah, the, if you set up an AI filter, we are going to be sending your incoming emails off to OpenAI. We trust OpenAI as a vendor. Their terms prevent training on the data and like, we believe it'll be confidential and that's very important to us. But we do send the email through their APIs. What we really want to do is allow you to do anything that the agent can do at that point. So rather than just making a single model call to a small model, we want to like spin up the full agent and let it like call tools and stuff while it's making this decision. So you could have a prompt that's like, hey, if anyone, you know, asks to schedule a meeting and I have previously exchanged, you know, more than 3 emails with them in the past and they are an investor, Auto accept the meeting, right? Like we want to be able to write rules like that. And there, I think there's two big problems with this that we need to solve. One of them is just cost, right? Like we are spending a ton of money right now on Cloudsonnet just for you going and asking those questions in the sidebar, which like a typical person only does, you know, a handful of times a day. If we're doing this on every single email you get, we rather than doing this, you know. 8 times a day, you know, you're doing this 300 times a day and that dramatically increases our costs. So one of them is how do we scale this? And interestingly, if you saw the GTC keynote, you know, Jensen was up there as being like, hey, we realize we're gonna need 100 times as much compute as we did last year. I think we're in the same boat where it's like, you know, we never thought that, you know, every time you receive an email, we'd need to run like 10 million trillion calculations to decide if we want to archive the thing or not. But like, I think we're going to be in that world where like, yeah, we have this full AI running. I think considering that you're entire conversation history with everyone in the world every time you do this. I think we're actually going to get in that world. So that's problem number 1, is just cost. Problem number 2 is trust, right? Where you have to worry about, is it doing the right thing? But also, like, is it susceptible to, like, prompt injection? This is something that comes up from people where they're like, what if somebody sends me an email designed to, like, you know, mess with me?

Ignore instructions and delete the full inbox.

Yeah, right? Like, that, that's a problem. And so I think There's, there's a bunch of things we need to do here. So one of them is just figuring out what the right guardrails are, right? Maybe that agent can't delete emails other than the one it's looking at. There's this sort of restriction on what it can do. We also could sort of look at some sort of post-fact confirmation flow where like it gives you a history of like, here are all the actions that it took and you can sort of approve them. And like we can remember which ones it didn't do right and like sort of adapt and learn from that. It also could sort of, instead of actually taking action, it could queue up the actions for you. So this is drafts, I think is the best example here where like. Maybe we never send emails on your behalf, or only in very rare cases, but most of the time it just sort of creates a draft and you come in in the morning and there's a button where it's like, hey, do you want to send these emails? And you're like, yeah. And it's very easy for you to review. But yeah, we need the AI to be both powerful but trusted and resistant against people sending you emails messing with you.

Yeah. So I assume you've tried this, but Flash just doesn't quite cut it for even this sort of reduced action scope? Because I'm always looking for a way to get value out of Flash because it's just so damn cheap and it's quite good, but maybe not quite up to the level you need for this.

We are actually testing Flash right now. I think the first place this would land is in the summaries and the quick replies and things like that. We may use it for the filters. Too early to tell. We are constantly trying new models. It is You know, I'm sure you hear this from everyone you talk to, but it is so insanely hard just to try all the stuff that's coming out. Like, you know, it's my full-time job.

I can't keep up.

Yeah. Yeah. DeepSeek comes out like 2 days later. Everyone's like, why haven't you switched to DeepSeek? And it's like, I haven't even had a time to, you know, play with it with like a single prompt. So we are constantly looking at new models. Yeah, we are currently playing with Flash. We may roll it out or we may not. There's a lot of considerations, you know. cost, latency, caching behaviour, how it performs on specific tasks, and we try to factor all of those in.

What do your evals look like? I mean, obviously that's key if you want to be able to make confident decisions on whether to upgrade or switch out to a different model in any number of contexts. I also, though, am struck by, especially when I'm looking at output that's supposed to be writing in my voice as much as possible and sort of representing me, that this is a very challenging thing to eval. Right. So I'm sure you must have some mix of objective and vibes, but what is that? What's the perfect balance between objective scores and vibes?

So we have made the conscious choice to say this technology is evolving super fast. Our product is evolving super fast. It is more important that we adapt quickly So that we don't break things and we move very quickly. So our evals basically consist of 2 pieces and they're very, you know, very seat of the pants. Piece number one is I have a Notion doc of golden test cases where I say, hey, here are some prompts that are supposed to work. I run those in my inbox. I make sure they do same things. So, uh, you know, when we're tweaking prompts, I go in and I try and be like, hey, does this still do the thing that I expect or do something reasonable? We don't want to lock it down to specific behavior because often it gets better. So for example, we recently added a tool to do unsubscribe and magically without us touching anything, anytime we did an inbox organization thing and we ran across an email that looked low quality with unsubscribe link, it would start offering to unsubscribe you. You don't wanna have a test that is like expecting organize your inbox to do a certain thing cuz that's better, right? This is, so you'll notice now if you organize your inbox, it's gonna start offering unsubscribe and that's— Something we didn't really think about when we first were building the feature. So one is we have a Notion doc and I go through it and I say, did this produce reasonable results?

The other thing, uh, how many prompts, by the way, just to calibrate myself?

All of 100 maybe at this point. Oh, wow.

Okay. It's a little more than I expected, but still not a huge thing. You run them manually?

Yeah, it's just a Notion doc. I don't necessarily run through every single one every time I make a change. Right. I'll go look at what are the ones that are relevant to to the areas that we touched, and I'll try those and see if they do reasonable things. The other thing that we do, which I think is the more objective metric, is we have an experiments framework. So every new change, every big new change that we roll out, we provide as an opt-in experiment for our users. And our users tend to be like super, you know, forward-thinking thinkers who like to play with this stuff. And they'll go and they'll turn this on like in large numbers. And we look at the retention stats of that. So like the unsubscribe feature, for example. We had like 99% retention for people enabling that. And when you enable that, it modifies the prompts and stuff and adds new tools. And like, it could, it could break all kinds of stuff. But, you know, after leaving this on for like a week and seeing like 99% retention on this feature, we're like, okay, clearly this isn't breaking a lot of people. People would start turning it off. So we feel comfortable rolling it out. So anything that is like a major change, we put into that form. We watch the retention stats over time. And the assumption is if retention is high, It's probably working pretty well for people.

Yeah.

Interesting.

Testing in prod, to put it lightly.

I mean, that's a society-wide phenomenon these days. By the way, when things do go out, is that also just something that you've made a strategic decision to live with? Like, there is no substitute for Sonnet, and so if Sonnet is down, we're down. Is it as simple as that?

Yeah, like we actually, you know, we could fall back to GPT-4o and it might give us decent results and whatnot, but there's always a question of, is it worth thrashing people to deal with a short downtime or whatever? So, so far, yeah, we've just sort of eaten the downtime and waited for stuff to come back up and we don't seem to lose users because of it.

Yeah, I mean, I think it's, this is something I've also been kind of, I don't know, it's a little bit of a hobby horse for me, even with, you know, my own company too. It's like, this is just the new normal. We're going to be more dependent on these services. They have pretty good uptime, but not perfect. And there's really not much we can do about it. If it's out, it's kind of out. So I'm with you on that. And I think just in general, there's a mindset shift that is kind of needed that's like, let's aim for creating really magical, the most valuable experiences we can. as often as we can and live with a little risk, whether it may be outage or just some uncanny valley behaviour. Certainly still see it, but I think it's for everybody, really. It seems to me it's worth taking a little bit of that risk to get those special upside moments, because when they happen, they're incredibly valuable. So, yeah, I think we try to be upfront with people about this.

And people ask me, like, hey, is this You know, is this going to be super reliable? You know, can I count on this? And my answer is, hey, if what you're looking for is the most reliable, stable product and nothing changes on you, like, use Gmail. That's what Gmail is. It's there. You can trust it, right? If you want the absolute most cutting-edge stuff in any email client ever, use our stuff. We are trying very hard to be at the absolute edge.

Yeah.

And you do have a really nice benefit too, that like, Worst case scenario, I can always just open up my Gmail and, you know, access stuff directly that way. One thing I did notice too about the new to-dos, I wonder if there's kind of a deeper philosophy underlying this, you know, so I go organize my inbox, the assistant, first time I did it, just like started coming up with to-do categories for me and suggesting like, okay, you should group these into this section. And I'm like, okay, interesting. That doesn't seem to touch back to Gmail at all, as far as I can tell. Then there's also labels which do, right? And I guess I'm kind of wondering to what degree you have found it to be advantageous to have this sort of single ground truth where something that you do propagates back into the core Gmail account versus— I honestly kind of like it better where I'm like, okay, this is my shortwave universe and I can let the AI assistant do its thing and run a little wild and worst case scenario, I can always go see my old view. But I don't know what users in general want. Do they want a unified reality or do they want you to kind of build something a little bit off to the side that in a sense de-risks them from anything that could happen to the core data store?

Yeah, I have a bunch of thoughts here. So the first thought I'll have is we Uh, one of the big sort of conceptual changes we made here since the last time we talked is when I talked to you last time, we were building an email client with an AI built in. We don't think of ourselves that way anymore. We think of ourselves as we are an AI with email features built in. And the plan here is to integrate in the medium term with products that aren't even email and basically sort of anything with an inbox, you know, your Slack or your LinkedIn or whatever, like you should be able to access And manage from this interface. And you'll notice we moved the AI to the left. Big driving force behind that is we see that as the main product. We see people coming in primarily because they want to interact with that AI. That AI can work with their email inbox. It can work with other inbox in the future. It can work with the CRM, your project management tools, you know, whatever you want to use. And so we're thinking it from that standpoint. And in that world, keeping everything in parity with Gmail doesn't necessarily make sense because you might not even be using it with Gmail. You might be using it with some other product. The to-do concept is here basically because we see a need for the AI to be able to, be able to like add and manage state specifically for like short-term organizational purposes. Labels are a great tool for long-term classification of like, you know, I want to apply this thing and then 2 years from now I want to go and search for this characteristic. And I want that label to be like short and simple and easy to remember. To-dos are great if. I have some project that's happening right now and I want a name that's like a whole sentence and I want to have a bunch of notes in there and I want to attach a bunch of email threads for it. So I want a much more complicated, but much more short-lived type of thing. And I think the AI shines with this, right? So you can, you know, let's say you're producing your podcast with me, right? And you and I might have, you know, 2 Google Docs and 5 threads about this and the AI can spot, hey, these are all related. I'm going to make one to-do that's like, You know, prepare for podcast interview with Andrew, and it'll put all this stuff in there. It can add some notes in there for you. And we don't sync that back to Gmail largely because there is no concept like that in Gmail. And we don't sync it back to labels because it's kind of a different thing. And I do think people like the idea that we are not trying to like shoehorn new features into old Gmail things in a way that like might mess things up. So a counterexample of this I'll give you and, and, uh, is one of our competitors. So if you use Superhuman, Superhuman implements some of their features basically by sending emails. And so if you use like the reminder feature or whatever, if you go back to Gmail after you stop using Superhuman, you end up with all these extra emails, which like kind of ends up looking cluttered and weird. And we try to avoid doing that so that at any point you can leave, you can come back to Gmail, everything looks the way you saw it before. And it's like an easy switch. Not that we ever want you to leave, but we want you to be comfortable that we are like doing the right thing with your inbox because it's, we know how important it is.

Yeah, certainly. I think it makes me more comfortable in taking a leap, and that is a big part of what people need to do, I think, to get value from AI, is be somewhat willing to take a leap. Couple more little product questions, and then the big shift, I think, is honestly super exciting, super valuable, because I have tried over the last 2 years, honestly, at this point, to fine tune a model to write as me. And one thing I've really learned is that if I want to make something general, and I've not really succeeded in this, to be honest, nothing has beaten dumping a lot of writing samples into Claude and just asking it to do the next thing. But I've learned some stuff by trying. One of the things I've learned is just that my— and I assume probably other people are often similar— no single system has the full me. Email. I don't like email back and forth with my wife. I would need text messages, really, to— that's where that kind of relationship in text form at least lives. And then Slack is key for a lot of the sort of day-to-day planning and discussion of what we're going to do. And then some of the more interesting conversations happen on Twitter DMs. And so it's like sort of not just that there's no single source of all the information, but There's honestly not even a single thing that I feel like really— it's very different facets, I guess, of my life, I feel like, across these different channels. So to be able to unify that into a single thing and have an AI system that could span all of them, I do think is quite exciting. When you do the Write as Me, how is that going? I would say I've noticed an improvement.

For sure.

It seems now usually that it handles routine stuff at least pretty well. So kind of classic, let's put a meeting together, here's my Calendly link, whatever, that kind of stuff. I'm increasingly feeling comfortable just hitting send on with the assistant's draft. Obviously, as it gets more specialised and more context dependent, it gets much harder. But what have you learned about making the write as me work maybe on multiple levels? What do people really care about? How representative or not am I? And techniques-wise, what is getting results?

Yeah. So from a techniques perspective, one of the things I am at this point, I very strongly believe, and I think this is a little bit controversial with other people I've talked to, but I very strongly believe this, is that The paste examples into Claude and have it use the examples to produce the, the response is the correct solution. It is the best solution. And that doesn't sit well with people. They like to assume that the more technologically interesting solution with the fine-tuning and stuff is gonna produce the better results. But I think there's two, I'll give you two reasons why I think this is the right solution. The first is this technique is much better at recalling specific facts As opposed to like a shallow fine tune. And when you're writing an email, or when you have the AI write an email for you, you don't care so much that it sounds like something you would've written. You care that it's correct, right? If you provide a link to someone, they're like, hey, you know, where do I go to like pay for this product, right? You don't care if the link looks like a thing that you would send. You care that it is the right link, right? If you, you know, if you're scheduling a meeting and it picks a time that you mentioned in a previous email, it, it shouldn't just be a time that you would send. It should be the right time.

Yeah.

And if you give it specific examples and frame those correctly where you say, hey, you know, the last time you talked about this topic, this is what you said, it can actually get the facts correct. It can get the links correct. It can get the times correct. So it isn't just sounding like you, it is, it is being accurate. So that's, I think, the point number one. Point number 2 is the big problem with fine-tuning is it slows down your ability to update models. And If you have built a bunch of fine-tuning infrastructure for a particular model and then you decide to change models, you got to re-fine-tune everybody. And that's a big process and that's a big migration. And the reality is we are switching models constantly, right? Constantly. And often the leading-edge models don't have fine-tuning features built in initially. And so if you want to stay at the cutting edge and you want to be able to roll things forward really quickly, you don't want to have to be fine-tuning these models. So we do have some fine-tuned models, but we don't do the fine-tuning for the purpose of Fact completion, and we try to avoid fine-tuning if we don't have to, and we, we do things this other way. Um, I'm actually fairly excited about reinforcement learning here because I think the big models do a pretty good job of sort of extracting facts and style and tone and stuff from examples, but I think they could do a better job. And I think reinforcement fine-tuning will allow us to teach the model how to do style and fact matching better in a way that is like generic across Everyone. So we don't like per-user fine-tune, you know, we just take the new GPT-4o, you know, with its reinforcement learning and we teach it, here is how to use examples for style matching and get facts right and write emails. So I'm actually very excited about that. But I do think the— here are 20 examples of things that have been said, match style and tone with the right prompt and the right reinforcement learning is the right— that's the right solution.

Okay, so how about on memory? I mean, I hear you on the operational challenges of fine tuning, and every time I raise this notion of per user fine tuning or even per company fine tuning, that's like, I think the most compelling counterpoint is like, what are you going to do every time you want to change models? And then there's also new knowledge that comes in and how often are you going to run it? And it does sound like a real bear. Is there I guess it still feels to me like there's some sort of missing middle in terms of memory where we have context window, obviously, and then we have stuff that is known in the weights, and then you have the sort of database call as well. But those all feel like they're still kind of missing a little bit of something. Again, I kind of come back to this intuition of, like, I know when I've found it, and there does seem to be something qualitatively different about that, or I know what I've tried. So I've been really interested in state space models. I've been really interested in the Titans paper that came out from Google not long ago, where they actually use a neural network as a memory module that gets updated at runtime. I've been quite interested in, you may have seen some, I did an episode on a system called HippoRAG, which was inspired by the hippocampus and how it is understood to connect concepts together. So have sort of an extensible graph network that they can query against. But maybe you think we don't need any of that and we just need better models and better search. And maybe this whole thing is sort of a confusion on my part where just keep pushing the current frontiers and it'll all work. And one day I'll just realise that we never really needed another middle piece. What's your expectation for that?

I, you know, I, I, I don't know what's gonna happen here. I, my, my spidey sense is there is gonna be some big breakthrough here. And there is gonna be some concept of memory that's sort of baked in that allows the models to be customized in a way that isn't quite so heavyweight as fine-tuning. But I have no idea what that's gonna look like. And I, I don't even have an idea of what that could be. Our approach right now is we have this thing called memories, actually as a feature in our product, which basically just allows the LLM as a tool to sort of manage a list of facts. And then we insert those facts into the prompt. And this is really useful for really for behavior customization. So if you want to be like, hey, every time I write an email, always CC my EA, or like anytime you schedule an email, like default to have it be 45 minutes or whatever, it's really useful for that. So we have that, but you have to be explicit. You have to say, hey, remember this fact and then we'll do it, or remember to do this. We also obviously. You search. I think, I still think there's a lot of opportunity with an agentic model to use search. You could have a database of, you know, interesting facts about you, and you could have a tool that like looks up the facts and tries to apply those. I don't think it'd be quite the same as like built-in memory, but you could probably do a lot of the same stuff. I have not had time yet to read the Titans paper. I think you mentioned that the other day. I'm very interested. But yeah, I do think there's gonna be some sort of breakthrough here, and I can't wait to see what it is.

It feels like one of the— it's too early to call a final frontier, but in terms of imagining something that could really work alongside me on an extended basis, it does seem to me like a sort of more integrated, dynamic kind of active learning, you might call it. But constantly updated memory does feel like something that would move the needle tremendously on that particular dimension. Okay, I think last question before we switch gears into the sort of meta of how you're doing it, because I think that's going to be really interesting too. But one thing we talked about last year I wanted to get an update on is we had speculated about the rise in AI-generated spam and sort of what happens as more people are adopting various tools and you've got AIs potentially talking to AIs. Has that happened? Another thing, of course, we sort of expected was like a lot of deepfakes during the election. That didn't really happen all that much. Is this another thing that we're just kind of all too worried about and it's not really a thing? Or are you actually starting to see any sort of interesting AI dynamics, let's call it?

Not a ton, honestly. I do actually get a lot of AI-generated content in support. Especially just like people being funny where like, you know, we'll send out a newsletter and they'll have it like write a poem or a joke or something to respond. And so people are doing that kind of thing. Um, but I haven't seen a huge uptick in AI spam. Our users haven't reported a huge uptick in AI spam. It's possible that the normal spam filters from Gmail are doing a good job, or maybe it's just not a big problem, but no, thankfully it doesn't seem to be as bad as I feared.

Okay, good. Yeah, I would say that. I still notice, honestly, more often I marvel at how I'm getting cold emails that were obviously written by a human and would have been better had they been written by an AI. So it seems like maybe just adoption generally lagging is still the broad explanation there.

A significant fraction of the emails that I send are at least AI enhanced. Some of them are fully AI written. And as far as I can tell, no one notices. I'll talk to people in person and they'll be like, they'll ask me this sort of question and I'll be like, hey, do you realise that my emails are AI written? And they usually say no. So maybe I just don't notice, which is fine if I can't tell.

Yeah, there's a couple of really interesting things there. I think I personally have not shaken this yet, but I've increasingly had to confront the fact that I'm probably just way too precious about the little elements of my style that feel to me like they make me unique. And I honestly kind of doubt that anyone else notices or cares. I've got a friend who really hammers on that all the time, and he's just like, nobody cares about that, dude. Your little flourishes, they're totally lost on other people anyway. I sort of also differentiate the world between routine tasks and non-routine tasks. As somebody who doesn't really have a job and is just kind of scouting the AI space all the time, I do very few routine things relative to, I think, certainly like your typical sales user who's got a CRM integration type of thing would need to do. But yeah, it's kind of philosophically challenging to be like, maybe all this style and all this sort of personality and all the care that I've kind of put into how I want to show up maybe doesn't really matter. And not only, it's one thing to think maybe an AI could replicate it, and it's another thing to think it doesn't even really matter if it replicates it. It's just do a good enough job and the world just keeps moving on.

But I really do think routine is the place to look for AI. Like I do a ton of routine emails because for example, yeah, we made some changes to the UI yesterday. I'm sending emails that kind of walk through why we did things and how to adapt your work style that are very similar between all the different people I'm talking to. And the AI is really useful in, you know, looking, hey, you just sent a bunch of emails kind of like this. Let's pull out some of the ideas here and reconstruct it to answer the question. But yeah, I'll send like 20 emails of like a very similar type.

So let's shift gears a little bit, if you're ready, to what it's like to work at and be building Shortwave. I understand that there is some news around a fundraise, and there's this expanded vision of not just email, but kind of omnichannel communication. And then I thought maybe the most interesting thing from your recent Twitter thread was, quoting here, we're also building an incredibly AI-forward culture. where the focus of our work is managing AI agents rather than making changes directly. We write a ton of code with AI, use AI for research and design, and even wrote this posting with AI. So I don't know how much you want to share about the fundraise, and you can expand on the vision. And then I really want to get into what's it like to be building this AI-forward culture and what's it like to be managing agents all day at Shortwave?

Yeah, so the company has changed tremendously in the last few months. We kind of reached a point where We figured out, hey, the future is not making a better email client. The future here is an AI that has your communication apps sort of integrated into it, but like that the main thing that people are doing in this thing is actually talking to the agent. And in that world, we're building a very different thing. It is not, you know, how do you streamline every interaction in the inbox and how do you make the UX of the email threads amazing? And like, we care about that stuff to some extent, but it is mostly How do we help people get things done at a higher level? And we're already seeing a lot of traction. Like, this is the reason we're doing this is because this is already where people are being successful. You know, our biggest and fastest growing plan is our most expensive plan that actually has, you know, basically like a bunch more money being sent to Anthropic for AI. Like, the people who are using us now, they care about the AI. And so we're just doubling down on that. And that vision, I think, has gotten our Like our investors are very excited. You know, they see, you know, opportunity for email client here, but they see opportunity for like this sort of thing here. And there does seem to be an opening in the market for an AI that can actually do things, right? If you use ChatGPT or you use Claude or Perplexity, like they're great at answering questions and doing research, but when it comes to actually doing work, they don't really have any capabilities to do that. And someone's going to build a thing that can actually get stuff done. And we see that opportunity. So we're starting to see our competition as like, you know, the next version of ChatGPT rather than the next version of Gmail. And so it's been a reframing. We have raised some money. I'm not ready to share the number, who was involved, but enough that we can significantly expand our team. So we're doing that right now. As part of this, we basically revisited everything about the company, you know, the way it's financed, the way we operate every day, who's on the team. And. We think this tidal wave is coming. We want to be in front of the tidal wave. I think the tidal wave is going to hit faster and harder than basically everyone realizes. And we want to make sure we're on the other side of it. And so we think it's going to require some different people with a different mindset to do what we want to do. And yeah, I think you mentioned the soundbite there of like, your job is going to be primarily managing a bunch of, you know, AI interns to get work done. Yeah. Rather than necessarily doing it yourself. And we're seeing this especially right now with code. If you're someone who is using the cursor agent mode, like, you know, a good, I think a good software engineer right now is going to have at least one agent running all the time doing something, right? They start the agent off and then they go start working on the next problem and it goes off and solves it. And like, we have found that it's possible right now. You can, we can take a bug report from Asana. We can copy it verbatim into that. agent mode and ask it, and it'll solve it in one shot in many cases, which is pretty amazing. So on the software engineering side, it's like, hey, we're looking for people whose skill set is primarily not execution. It's not writing code. It's primarily understanding the problems, understanding the components involved and how they should interact, and being able to like frame those prompts. And so the value has shifted from people who can get things done to people who can understand the problems and structure things. So that's on the coding side. On the design side, I think it's changed too, where it used to be that the way that you design a product is like, you know, you get together in a room and on a whiteboard and you draw it out and then you make some mocks and then you make a prototype and then you build a thing. And now the first thing you do is make a prototype, right? Like before I even talk to anyone else, I just build it in Bolt.new. And it's only after I'm like, yeah, I think this will be pretty good. Then I go and I talk to the right people and we like make a mock and we try to steal figure out the details, but it lets us kind of cycle through bad ideas like much, much faster. So we're looking for, you know, on the design side, we're looking for designers who have already made the shift of, I don't start with a mock, right? I don't even start with a wireframe. I start with a working version that I've built with Bolt.new or a tool like that. That's the beginning. And so our design, I think, is totally different. Same thing with, with like content. I mentioned like the job postings that we wrote. That's all ChatGPT-4.5. It's really good at writing, right? It's not, it's not quite at the level of like if I had really put in effort and done a good job myself, but it's not far off. And I can produce, you know, content that's at a pretty high bar in a fraction of the time. And so, you know, we need to do a lot of comms here. We want people who are good at, you know, how can one person manage our blog, our social media, our changelog, all the docs on our website, everything? How could just one person do this if they're thinking hard about the right prompts and the right way to use these tools and how to have it generate this stuff? So it's gone from, hey, maybe we need a team of 50 to do this, to maybe we just need a team of like 15 or 20 people with like the right skill set and the right usage of these tools. And, you know, the 30 people that aren't there anymore are really more on the execution side, and the 15 or 20 there are really more on the sort of like problem understanding and solving side. And I think it's going to be a big shift. And we actually, I think I mentioned this earlier, but we actually let a couple of people go who were really very talented, very good at what they did, great employees, great attitude, but not quite right for this new world that we felt we need to get into and not super passionate about it either. And so we thought a change needed to be made.

Passion is super important.

I mean, that's definitely something I observe all the time. It does take some curiosity, just innate interest. I feel like to the degree that I have demonstrated some aptitude here, a lot of it really just stems from that. Is this something, I guess at this point, is this something that you think is an investment in the future? Obviously it's that, but is it also something that you think allows you to be more productive or execute at a higher level today? Or do you feel like it's kind of like, well, we might not be like you mentioned with the blog post, maybe it's not quite as good, but it's fine because it's pretty good. And we're also investing in sort of being the type of organisation we think we have to be in the future. Does that make sense? What is the time horizon for ROI on this?

I think we're already seeing it in a big way. And I think companies need to be making changes now to the way they work, the way their org charts work, the way they spend their money. I'd say, you know, I don't know if I can put a number on it, but our engineers today are significantly more productive now than they were 6 months ago. It's really changed in the last few months. Like if you last tried this stuff last summer, the world is different. Give it another try. So significant differences there. On the design side too, like, yeah, we can just skip over half the process and get right to the prototype. And it, I wouldn't say it, you know, changes the work needed to build the final design dramatically, but it does allow us to cut out a lot of bad iterations, which is where most of your time is spent anyways. So I think we're a lot faster there. On the content side, also a lot faster, right? Like, good writing is hard, and most of the time you don't need to write really amazing writing, you just need to write pretty good writing. And, you know, if you can do that much faster, you can keep your docs up to date faster and things like that. So I think we're seeing those gains today. But there's a related thought I wanted to add in here, which is a big change in the way we're thinking about our business. So a year ago when I was talking to investors, I was saying, hey, their question was basically like, what is your moat, right? Like, you're building this cool AI stuff, and but why can't someone else come and do this? And I was like, well, our moat is that we have an email client, and it took us years to build this thing. If anyone else wanted to enter the space to build like an AI-enhanced email experience, they'd have to go build an email client. That's really hard. And now I think, you know, The thing that took us 4 years to build, it might not take 4 years for the next person, right? It'll still take them a while, but they're gonna be able to do it a lot faster. And I think there's this big change happening in the industry where all of these companies that have these moats of like the software they have built, that moat is being significantly eroded. The value of that like code that's sitting around is going down very quickly, and they're gonna have to come up with some other moats. And we're looking at this and saying, hey, the moat that matters, the only moat that probably matters is speed. And so how do we optimize our team for speed? And I think the best way to optimize your team for speed is to keep the team small. It's just harder to get consensus on big teams. So we're like, okay, how do we have 15 people that are the right people with the right tools that work together really well so we can just like move, move, move. And our moat is that we're always ahead of everybody by, you know, 2 months. Because of the way we're structured. So that's been a big change in our thinking.

So 50, is that like a long-term number?

No, not a super long-term, but I think for the next year, right? Like, I don't think we're gonna go beyond 15 and then we'll see beyond that. But no, I don't think most people could build the product that we're— the scale of the scope of product that we're building, you know, historically with 15 people, but I think we can.

Yeah, I mean, there's a lot of surface area already, and it sounds like you're planning to add a lot more. And on top of that, got a lot to keep up with.

Well, and we're not, you know, we're not the first ones, I think, to figure this out. Like Cursor, you know, it's like 20 people. Midjourney is like 20 people, and they're obviously wildly successful. So I think, I think there's a new era.

Are you still subsidizing users on the margin? That was another, um, tidbit from last time. You said you were literally like losing money on every Is that still true?

Uh, not anymore. No, we've done a lot of work to, to kind of increase efficiencies. We've also added like a higher end plan and we've gotten people to kind of opt into those higher end plans. So we are, uh, margin positive. I wouldn't say hugely margin positive. We still spend a significant fraction of every dollar that you give to us, like goes directly to LLMs. And then a lot of that also goes just like traditional email infrastructure, but on the margin, we do make money now. Yeah.

All right, that's cool. Is there like a 2 to 10x more expensive version of the product that you could imagine rolling out? Or are you already basically— I mean, you're not quite maxing out, right? Because you talked about you could spin up the agent every time. Could you do that today if you just charged 5 times more?

We could, and I think we probably will. If you look, I think I mentioned this earlier, but we have all these different plans and The, all the growth basically is happening on our most expensive plan. Everyone's coming in and they want the most expensive plan because it lets you have the biggest context window and it lets you index all of your history for search. And people care a lot about that. And even though the, you know, our business plan can give you almost as good answers as the Premier plan, but people want the best answers. And, you know, the difference between good and best is worth a lot of money to them. And, uh, yeah, I think we undershot. I think. You know, ChatGPT Pro came out at $200 a month. I pay for it. I think it's too low.

No hesitation on my end either.

Yeah, I, I would, I would pay more than that. No, no hesitation. And, um, I think we should probably do the same. I think there should probably be, for the type of person who really truly lives in their inbox, who's able to get the kind of value out of what we do that we want them to, I think we could also have a $200 a month plan. Um, and I don't know how far this goes. Like, I know Sam was Talking about like, maybe we'll have a $2,000 a month plan or a $20,000 a month plan. And I have no immediate plans to do that, but to the extent that people want us to spend money on GPUs on their behalf, like there's, there's sort of no limit here. So for example, right now we don't, I think you mentioned like the automatically running the full agent on every email. That's a great place where we could spend a lot of money on your behalf if you want us to. And it could be a— 100x increase in the amount of compute required. I think another example is reasoning models, where right now we don't use reasoning models. They're much more expensive, they're slower. But if, you know, you're trying to do something complicated, like I want you to give me like a detailed analysis of every customer report over the last year, right? You know, this might be something that otherwise you're giving to an employee and they're spending a month on, and you might be happy to spend, you know, Hundreds of dollars of compute just to answer that one question. So there may be something that we can do there. So, yeah, reasoning models, automatic execution of things, bigger models in general. And yeah, maybe there's a $200 a month plan or maybe more someday.

Going back to the building side, are there, you mentioned Cursor, of course, and I'm a Cursor user and I've also gotten into Bolt and to Lovable and basically try everything I can. What I haven't quite figured out yet is how to make things work together, how to effectively sort of architect systems of agents. And Replit too, right? Great experiences with all these different things in their moments. But do you have kind of a mix of things that you've brought together? Are there things that sort of complement the coding agent. I'm thinking, for example, of a company called Codo that specifically emphasises testing, or there could be observability or sort of monitoring tools. I mean, monitoring is another thing that you could really imagine layering on, and it ultimately seems like we're going to need it, right? If you're going to have 15 people who continue to run at roughly human speed with a lot of AI assistance, they're also going to need AIs to sort of help them supervise the AIs, right? I mean, as the pyramid built under each person and under the company collectively gets bigger and bigger, there's got to be a whole architecture and sort of specialisation within these agent ecosystems, right? So I guess, is that roughly how you see things shaping up? And have you started to tie these things together in a useful way at all so far?

Yeah, I have a couple of thoughts here. So the first thought is, I think one of the potential futures for Shortwave is really as sort of a routing layer for the messaging going on in your business life. So you might be using a bunch of these different AI tools, but something needs to sort of like take the incoming events and decide which agent handles it and get it to that agent and take the output from that agent and do something sane with it. And inboxes are great great tools for sort of managing that flow in a way that both the human and the AI can collaborate together on it. And, uh, if we're going beyond email and including other kinds of human communication, maybe we're going to go beyond traditional human communication. Maybe, maybe you're going to start thinking of us more like a, like a Zapier type of thing with like a human UI that you can use as well. Um, and we might be a routing layer that helps you route things into these other tools. So we've had requests from, from customers like this, and this was a discussion I actually just had with one of our investors. Of, you know, maybe we need to start thinking ourselves more as like a, like an agent routing layer. So that's my first thought. My second thought is we have tried in our approach to agents some different ways of having like a multi-agent approach to solving problems where basically like you, you know, you have one system prompt that you iterate on over here and then once you decide it's a certain type of problem, then you like change the system prompt, you add in a different prompt, or you hand it off to a different model. And our experience has been so far, and you know, this is just one experience, but our experience has been so far that those systems end up being kind of brittle. They tend to struggle at reasoning across different types of tasks. And the cost aside, the better approach tends to be just take the biggest, most expensive model you possibly can, stick all the instructions into the context, and let it reason across the different things. And That, I'll give you a specific example of a win we had here recently. So we used to have the ability for you to add custom instructions for certain types of operations. And the way that worked was when, for example, for writing, you could add like custom writing instructions. And when we call the tool that was used to grab the writing instructions, we would insert those custom instructions into that thing. That worked well for cases where the writing tool was being called appropriately. But if you had some custom instructions that were like, you know, trying to give it hints of like when to write emails or what types of examples to look up, or that wanted to handle like not just things during writing it, during other times, it didn't work so well because these things only got plugged in certain times. And what worked much better was this memories approach where you basically say, hey, we're going to take some customization instructions, we're going to add them into the master prompt. And we're gonna include that in every call. And then you can have a customization that can be considered at any time, right? Like you could have a customization that's like, you know, always address Nathan Labenz as sir, right? And it could do that when writing emails.

Please.

It could do that when scheduling calendar events. It could do that in any situation. And that has proven to be like a much better user experience. It produced much better results from the AI. And so we, yeah, we used to have a model that was sort of like routing things to different AIs and just the screw it, we're just gonna put it in the main prompt has actually worked much better.

Yeah. More cache friendly too, I guess. Right. To have that consistency.

Yeah. So I see the opportunity for kind of like the multi-agent world really being more about working across organizations where, you know, we need some interface between like our team and the cursor team, but probably not something that's happening like within our app. I think within our app, the approach of just like one the biggest LLM we can get and the biggest context, and then using caching really well. I think that's probably more likely the future for us.

So I share the same intuition just based on everything that I've tinkered with over the last couple of years. And I just actually told a company that I have been doing a little bit of agent advisory with that. And then 2 days after I said that, OpenAI came out with their thing where handoffs between agents is a pretty notable new feature.

Is that—

what do you think is driving that?

I literally talked with that team yesterday about that exact thing, and I had the exact same questions for them, and I shared my perspective. And I think— I don't think handoffs are going to take off. That's my hot take.

Yeah, I mean, I can see some upsides to it. I mean, the things that they cite are like, it's easier to test them in isolation and whatever, but Yeah, I mean, it's tough when you then have classifications and yeah, there's just more things that can go wrong, that's for sure as well. Okay, well, I'm glad to hear somebody else shares my intuition on this. It makes me feel like, because I was like, am I out of touch on this? Certainly OpenAI has some real insight into what's going on in the broader world.

They shared the same perspective with me around sort of developing things in isolation, but I think the reality is with an agent, you don't really want the tasks to be isolated. You want the agent to reason across all of its capabilities and make a smart decision. And maybe there are some cases where you really want to sandbox it, but my experience is that's not usually what you want. You usually want to think about everything it's capable of and make the best choice. Yeah.

I wonder if this is to some degree driven by larger companies that sort of have existing organisational structures and who's responsible for what. And those lines are really hard to cross. I heard one of the more interesting little tidbits that I've heard in AI discourse over the last year probably was with Yi Tei from, I think, the Latent Space podcast, where they were talking about multimodal models and why they've developed as they have and why it was like vision models and language models developed separately and then kind of merged with these sort of late fusion approaches. And his take on this was very simply, that's a reflection of legacy team structures. Like, it used to be that you'd have a vision team and a language team, and so they would do their things, and then maybe you could try to fuse them. But the upshot was like, that won't last into the future. You're going to have just unified teams and you'll have unified architectures. From the beginning. So this feels like maybe the seed of kind of a similar insight where it's like, maybe one reason, because I've often also been like, why? We talked about this a little bit last time too, with respect to Gmail specifically. Why can't the big incumbents do a good enough job with this? Right? The technology is not that hard to use relative to— it's one of the things that's so great about it, right, is it's so flexible and in many ways forgiving. And Totally understands what you meant, even if you're riddled with typos. I've even seen people doing this speed typing where they just put tons of typos in and have the AI correct it. And for whatever reason, I haven't gotten over the hump on that myself. But maybe there's something here in terms of who will win in different verticals that is just like big companies with these org charts and these division of responsibility and who's going to sign off on what. Maybe these things are being built for them because that's what they kind of have to have to be able to make any sense of it. And maybe it's not really about what's going to make the thing work best. And an approach more like yours where it's like small team, totally unified, single agent that knows your whole history and can kind of deal with you on that basis. It does seem like what I want. I don't want to be— it seems counterintuitive. I don't want to be handed off from one AI. I don't want to be handed off from one human to another. I don't want to be handed off from one AI to another. So yeah, this is interesting. I think there's definitely some, some insight there that might prove predictive.

I think you're probably right. I'm sure they did this in response to like some real needs that they're seeing. But, you know, I think one of the reasons we started Shortwave back in the day is, you know, worked at Google. I had some insight into like what sorts of organizational struggles they were going to face. And I had the belief that the Gmail team was not going to be able to You know, innovate quickly for organizational reasons that were very hard to change. And I think that has borne out. I think you're seeing that now where, you know, the models from Google are awesome, right?

The new 2.5 is literally mind-blowing.

But like the, the product progress in using those models in interesting ways in in stuff like Gmail is way behind. And I think that's driven by— I, I just thought that I don't think these people are smart or whatever. I think it's just driven by the way the organizations work. And, you know, for example, we decided, hey, we need that, we need the LLM on the, on the left, right? Agent's gotta be over here. There's good reasons for this, right? If I was at Google and I was like, we need to take the right sidebar and put it on the left, that'd take me 2 years, right? Like, think of, think of all the sign-off I need to get and all the people I need to get to Convinced to have buy-in here and all of the, you know, you literally have 1,000 meetings, right? It would take, and for us it was, you know, it took a few weeks, so there was a lot, but it was all the discussion was, is this the right thing for the product? And we could focus entirely on that question and we can just move a lot faster. Yeah.

So going back a little bit more just to who you're looking for and, you know, what the, you know, what the hiring process is like. One thing I noticed is, as far as I can tell, all the roles are in person, San Francisco. And I have kind of a question about that, especially as you think about a future in which you have a lot of the work being done by AIs. In a sense, it feels like the AIs are always remote. And personally, I have had, being in Detroit, it's like quite different local talent pool, and the pros and cons are quite different. But I've had a great experience hiring remotely. It's definitely opened up the aperture of who we could go for. And I kind of am like, well, geez, if all the AIs are only going to interact with me through a screen anyway, at least until I've got humanoid robots sitting at the desk next to me, it'll be that way. How are you thinking about holding the line on in-person versus potentially liberalising to remote?

There's a couple of pieces here. So I think Piece number one is what I talked about with speed. We have been historically a remote team. In fact, at one point we were fully remote. We weren't even hybrid at all. And I think it's becoming increasingly important that speed is critical. It is a lot easier to move quickly in person, especially when what you're doing is making like rapid product changes, right? You can, you can move in a straight line very, very quickly remote, but if you need to like quickly change course and have a lot of difficult meetings and conversations and have to deliver tough feedback and be like, hey, this isn't working. It is just a lot more emotionally challenging to do that in a remote environment. It's possible and teams do it, but I think it's harder. And I think it's something we really struggle with. So if speed is paramount, we think having a core team of folks who are driving most of the roadmap, the product decisions and stuff that can meet in person and hash it out on a whiteboard at any time, I think is really key. So we really want to have that happen. The other, I think, really key piece is, is just, it's just a reflection of the founders. So Johnny and I work better in person. I am a better leader and manager in person. People, I think, like me better if they see me in person than if they're talking to me over a camera. And that, that's a weakness, probably, you know, like, I'm sure I could learn to be more effective in a remote way. But I also, to some extent, need to, you know, understand my capabilities and limits. And, you know, I look back at Firebase. Firebase was a fully in-person company, and there were a lot of struggles that I think we've had with Shortwave that we didn't have there caused by some of the differences there. So I think Johnny and I looked at this and said, hey, we need this to be a place that is optimizing for our strengths, and we work better in person, that we need to have a core, and we need to move really, really fast. We gotta have this core team in person. And yeah, I think this can make recruiting harder. There's lots of super amazing talent that is not in San Francisco.

Yeah.

I'm very keenly aware of that, but we think it's worth it.

Yeah. Okay. How about an AI scout role? This is a hobbyhorse of mine. Have you thought about a dedicated position for just like somebody to go try every new thing, whether it's new model, new framework, new agent experience, whether somewhat competitive or totally far afield? This to me feels like something more and more companies are going to need, but I don't know how many are feeling it quite yet.

Are you asking for a friend, Nathan?

Yes.

Well, many friends, actually, I think, because when you would have been 50 and you're now 15, and that starts to get generalised, we do need some of these new AI jobs. I've managed to kind of stumble my way. Sometimes call myself the Forrest Gump of AI, where I just sort of unintentionally stumble through these important scenes and I've ended up in a place that I honestly quite love and have nothing but appreciation for. But I do think we need a lot of new AI jobs, and this feels to me like one that might become common. And so, yeah, I think I'm asking on behalf of maybe a lot of people.

Yeah, 100%. It's not a role that I have listed right now just because I think we're a very small team right now and we only have a handful of roles. But I do think it is sort of impossible to keep up right now. I do think there have been a number of moments where our discovery of, for example, if I hadn't listened to that particular podcast that talks about the new behavior of Claude, it might've been months before we figured out the same thing, right? So I think there has been a number of inflection points driven by discovery of new technology that I think we just got lucky on. It'd be great to have someone who's like systematically doing this sort of thing.

Yeah.

Obviously, I try to listen to great podcasts like this one and keep up that way if I can and read whatever I can. But yeah, I think in the future we may very well have a role like that. And I think a lot of other companies, especially bigger companies where not everyone is constantly thinking about AI, could benefit tremendously from something like that.

Yeah, situational awareness, harder and harder to maintain all the time. Maybe last little bit on the hiring side. To some degree, obviously these are just labels, but I did also notice that the roles that you do have posted are all staff or senior designations. I guess maybe 2 layers to the question. One is, like, is there a place for somebody who doesn't have a lot of experience at Shortwave? And what would it take for them to take maybe a somewhat extreme case? Like, let's say they're freshly graduated, no work experience. Is there any way for somebody like that to demonstrate to you that they have the skills that would make you even open to hiring such a person.

Yes, absolutely. And for what it's worth, the job postings have been updated significantly since I think you saw them. So I posted the senior roles first. We actually have some entry-level roles up there now. Oh, okay, cool. Including, I just posted yesterday, we're looking for someone for customer success. We're looking for like a content creator. We want to make a lot of videos. Because I think there's a lot of, just a ton of sort of like discovery and education in AI. So we want to do some content creation. There's a junior kind of product engineering role up there. So we are, we're definitely looking for some junior folks as well. If you want to impress me, there's a very simple application process, which is send me a video. We've kind of gotten rid of some of the sort of take-home tests and things like that because they're too easy to game with AI. So we're going to just, you know, send me a 5-minute video, show me something cool that you did. I think the thing we're most interested in is Are you someone who's like really forward-thinking with AI? Have you figured out, like, if you could show me, here's a way that I leverage AI to do something useful in a way that I haven't thought of and impresses me, I think that would be, you know, that would cause me to take notice. So yeah, are you super forward-thinking with AI? Are you being creative about how this stuff is used? Are you staying on top of it? As you know, if you can stay on top of it, like, you're pretty impressive already.

Yeah, no doubt. I guess more broadly, what do you think is going to happen to the future of the software industry? This is obviously a super hot topic right now. Are there going to be more developers because they're more valuable, or is there only so much software that needs to get written? And also, this could definitely break down in phases where I think we might be on the up ramp of that curve. But the more I hear from folks like you that you only need 15 people for the foreseeable future, the more I'm like, Man, I don't know how much we can bet on this sort of developer market to grow or even sustain itself as it has been for the last few years. What's your expectation?

Yeah, good question. My co-founder oscillates depending on the day between an existential crisis of, you know, he had the CTO, right, of like, I am obsolete. And on the next day, he's like, I am a god and I can do so much.

And he doesn't know which one to take. Right?

So, uh, I think, I feel like every software engineer is going through that right now where like their productivity is like going through the roof, but also maybe I'm obsolete at the same time. I think it's going to change a lot. The nature of what you use is going to change a lot because, for example, like actually writing code, like localized code, is starting to become something the LLMs can do. So for example, like any sort of front-end development where you're like, hey, I need a button that looks like this, right? The LLMs are great at that sort of stuff. And so I think increasingly code's gonna get written by this. There's a, there was a, uh, interview with Dario from Anthropic who I think he said like within 2 years, like 90% of all code is gonna get written by, or maybe it was 100% of code is gonna get written by AI. I'm not sure I quite believe that, but I think it's gonna be very significant amounts of it. The places where I think AI is gonna take longer to, you know, if ever to, to be able to do it, is really sort of understanding user problems. and understanding the components and how they interact, which I think is super important, super complicated, and actually a huge part of the job today. If you take a senior engineer today, the senior engineer isn't senior because the UI code they wrote is dramatically better than the junior engineer. The senior engineer is able to think about the whole business problem and solve it. And so I think The folks who are good at thinking about the business problems and how to solve them and thinking about the components and how they interact, I think they're gonna do great. I think the folks whose strengths are on execution side are gonna have to learn some of those other skills and get good at that. In terms of numbers of people, I don't know, right? On the one hand, yeah, I think you can do more with less. On the other hand, suddenly there's, there's a million little companies that should have existed, but it's just too expensive, right? Like software engineers are crazy expensive right now, and if you need 10 of 'em to do anything, right? But suddenly there might be a startup where, you know, you would've taken 20 people before, and now you can do it with 3. And, you know, you can build a niche product for something. So maybe we'll just see an explosion of, you know, another 10x explosion in software and, you know, the jobs will stay the same. So I don't know. I'm generally an optimist. I generally think that it will make life better for everyone, including employees, and it just, everyone's gonna have to adapt, but I think it's gonna be a good thing. I think being a software engineer is gonna be an awesome place to be over the next few years. Yeah.

Yeah, I can't imagine coding without AI assistance at this point. So on that level, it's no doubt. And I definitely share your expectation on 10 times more software getting created. And I'm still kind of like, but maybe even 10 times more software getting created isn't enough to sustain the number of just pure headcount jobs that we currently have. So time will tell.

Yeah, for sure.

I do advocate people start thinking about universal basic income if they haven't already started to contemplate it. But we're not quite there yet. That's like a long-term problem, like maybe 2, 3 years out. This has been outstanding. Maybe one last question. As you look a little bit into your crystal ball, what do you think are going to be the big developments for, say, the rest of this year? What are the things that you're kind of Man, if only they could get this to work or fix this. If only the world were a little bit different, things would be much better or much different for you. What are you kind of most closely watching in that sense?

Yeah, I think the number one thing is post-training on agenda behaviour and tool calling type stuff. So the step change between every other model before Claude 3.5 and Claude 3.5 just enabled a whole bunch of new stuff. 3.7 was better, But there's still gaps. It'd be great to have more competition there. Great to have OpenAI and other folks have options here. And I think this is all coming down to agentic-specific post-training. And I am excited to see where that leads and how good that gets. And like iteration and tool calling can work around so many limitations in your system and the AI can sort of find creative solutions to stuff that I think the sky's the limit there. So that's number one. Number one thing is like. Post-training on adaptive behavior. I'm also just generally excited about improvements for the productionization of these things, performance of the tools, cost, latency. There's a lot of things that we can't do because they're impractically expensive, like running the full agent on every email that comes in or adding more search results into every query. And the cheaper, faster, more reliable these things get, the better for us. And I expect that to keep happening. Cost is still a huge factor for us. So that's another one. I think another area is kind of more native multimodal voice. Um, today you can do voice things. We actually have a voice input in our app. It's fine. It's not great. It's not like really talking to a human, but models that are more, you know, support voice natively, multimodally that we can use for the full agentic flow would be like totally killer. So fingers crossed there.

Yeah, I love that vision too. I mean, anything that will get me outside more and less tethered to my desk is hotly anticipated, certainly by me.

100%.

Any other thoughts or closing wisdom you want to leave people with?

Just, you should come check out Shortwave and you should totally either apply yourself or refer your friends. We have a $10,000 referral bonus for Anyone, if you get them to apply and tell us that you referred them, we will give you $10,000. No joke. So yeah, help us find great people.

Love it. Andrew Lee from Shortwave, thank you again for being part of the Cognitive Revolution.

Thank you, Nathan.

It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at tcr@turpentine.co Or you can DM me on the social media platform of your choice.