Notion Agents Update: The Framework, The Cost, The Payoff

At the end of the day, if you look at work as inputs and outputs, almost like a factory, that's where you install custom agents. Think about what do you need out of it? What does it have available to it? And, uh, and then what are you going to ask it to do to add value, to go full circle? Welcome to the Notion State of Mind podcast. I'm your co-host, Josh Redd, joined by my co-host and co-founder Bryan Takayama. And today we're going to talk about agents, not the, not the men in black that show up at your door after you post something online you thought nobody was watching, but the ones in Notion that help you work better. So Bryan, what, what, what do we know about agents so far?

Well, I think we're at a really exciting time right now where, you know, alpha came out back in August of last year, but they came to GA just a couple of months ago in February. And they have been free for everyone to use for the past 2.5 months for people to develop tens and tens of thousands of agents across their workspaces. And this is really top of mind now because they are going to start costing money. So we are at the point where custom agents are full GA. They are being deployed everywhere. And now we need to start understanding not just the value that they're driving, but that value relative to the cost that will be aligned to them.

Yeah, about by the time this podcast comes out, they might even just be, be out there in the wild. Uh, so just to give you some context about, uh, where we're leading up to and, and, um, hmm, I want to tread lightly here. Is there a bit of a strategy being undertaken in, in Notion's part, uh, by letting people kind of go wild with custom agents and then flipping the switch? I mean, let, Let's be real, their business can't give us free lunches forever, right? So what's, what's the strategy here and how do you think that our clients or Notion users should react to the reality that they've had custom agents for free, now it's going to cost, and we can even talk about how it's going to cost them after that.

Yeah, it's a really, I mean, it's a really top of mind topic based where we are right now. And it's kind of a hard one to. Evaluate at the moment. Um, we're building a lot of tooling internally to give us a better way to evaluate it, but Notion was just a per-seat cost. Now it's a per-seat plus a usage-based pricing cost. And that new component there is really important because what that, that total price tag ends up being is n right now. We have no idea what it's going to be. Um, right now it's like $10 for every 1,000 credits that are being used. Uh, and it's Every run based on the complexity in the model that it's using is a different amount of credits being used. So I think what we are working on right now is figuring out, one, how do we control what, what agents are being created? Two, how do we create a tool that can evaluate the costs associated with that? But more importantly, how do we create a framework to evaluate the value that the agent is creating? That way we are not just applying new costs to IT spend, but we are being able to say this is the value associated with the IT spend.

That's coming from the usage-based pricing.

So yeah, there's something to be said about kind of the new frontier and the people who are on board, who see the vision, who see the future. They're willing to take on the cost even when it's unknown, right? The cost is— that's probably the biggest issue with AI for business adoption is it's a black box. And when you want to sell that to somebody who cares about dollars, and they should, that's a really hard sell when we're still just connecting, you know, a bit of like technological capability. We're trying to still connect that to real value delivery, to real ROI. And that's even something that we're thinking about internally is like, we're not trying to, we're not trying to sugarcoat things, right? So we're not trying to just pretend like it doesn't cost money. It does cost money, but you should think about the cost more in terms of value. Like in the same way that a person costs money to hire, you don't hire people without expecting some sort of, you know, ROI on the kind of work that they do. So how do you think about that in terms of agents? Is it kind of one-to-one? Like, should you think about agents the same way that you think about hiring on people? And it's kind of up to you on onboarding and setting them up for success. Like, how do we, how do we step into this world also knowing that it's like very frontier still?

My, my gut says, well, it, it's kind of funny because it's just a little different when it comes to a plat—

each platform.

I have no problem putting a user into an Anthropic Claude seat for a monthly pro package, or if I'm on enterprise to pay for tokens because they're getting access to a chatbot and it's really hard to kind of measure what's happening there, but we assume it's value. Um, when when it comes to a custom agent, it's like, it's quite different. It feels different. And I don't know why it feels different, but it's like we're creating this, this tool that's meant to either help one person or help many people. And it takes intentional effort to create it and it's really hard to monitor. So when I'm thinking about how clients should be thinking about custom agents, I have to flip a switch in my mind to be able to talk about it differently than I would with any of the other platforms.

Yeah. Something I've been thinking about lately and I was even Texting some friends of mine, they own a web development business, you know, and AI, especially, you know, things like Claude Design coming out. I've been thinking about this idea of like your business and even the people inside it, they fill gaps. There's a gap between what somebody wants and what they have access to or what they can get. And you fill that, right? That's kind of value creation. At its core. And if we're gonna get philosophical about like what work is, like it's value creation, right? So you're filling gaps, you're, you're providing value, and that's why people pay you. That's why you have a spot in a business. And with AI, obviously it's compressing a lot of, of work and a lot of those, those gaps that were being filled just because they were so tedious or so industry specific or they required a ton of knowledge. A ton of, uh, of a certain skillset, those are being massively compressed, right? But there's always going to be gaps to be filled and just a matter of like being thoughtful about where the gaps are and, and filling those gaps and remains to be seen, you know, after, after a lot of different areas get compressed, but remains to be seen, like there will always be room for, okay, what's the next thing that I want that I'm going to look for? You know, that gap to be filled. I'm going to look for somebody to fill that for me, or I'm going to step into that gap. I'm going to find somebody to fill that gap. I don't know. Does that translate?

Yeah, I think that's why people hire, right? It's like, I need somebody to do this thing for me. There's a gap in my ability to access this information or my ability to run this analysis, or I need somebody to help coordinate these activities, whether it's across different pillars in the organization. Yeah, I think that that's, that's exactly right. And what we're, what we're looking at with the broader AI area now is the compression of it and Notion AI custom agent specifically is the ability to inside of Notion natively be able to start to fill those gaps for people in a much quicker fashion.

What is the value proposition of using Notion AI versus other AI tools?

What do you mean?

Well, one, I think it's a given that your AI token price, like your per token price is going to be higher by, by doing it through Notion, right? Because it's going through their kind of bundling of credits. And then, uh, they've got to have, they've got to, you're kind of using a middleman in a sense. But what is the value of, of having that middleman? What is the value of using Notion compared to setting it up yourself or, you know, using an external tool?

When you say Notion AI, then you're speaking specifically about custom agents, not Notion AI in general.

Yes. I mean, Notion AI does have a cost as well.

It's your seat though.

And it's like, you can, yeah, it's like it's packaged now. It's like it used to be an add-on, now it's packaged in Business Enterprise. Right.

But, and it's super inexpensive relative to the other platforms. Yeah. And you get access to all of the different models. Like, I can't go into Claude and access CodeGPT-5.4.

Yeah.

Or Gemini 3.1. Right. Like all of that's baked into, into Notion AI, which is really useful.

So Notion AI in general, this is probably where we're landing, is a very cost-effective use of AI, like just having it at your fingertips. It's that the cost is known, it's per seat. Whereas now when we step into the custom agents, especially as they flip the switch on cost per token, what kind of areas do you think that it'll still be valuable to use Notion custom agents in particular? Versus having people use maybe more token efficient methods.

I guess just to question your question, why, why would you say custom agents are not token efficient?

Based on the idea that they are, you can go direct to the platform that provides the tokens with Claude or ChatGPT or whatever. And because you're going to get them through Notion, you're gonna, one, they've gotta, they've gotta have their, their margin, but of course they've got, they've got bulk pricing. Uh, but let's just use the concept of you're, you're kind of going through a middleman to use the same tool, but it's going through the Notion harness. So there's a value there, right? Plus the non-month rollover, you're gonna, you're gonna burn. Like, let's just face it, like you're not gonna be perfect at that. And that's where you're gonna, that's where you're gonna lose. You're gonna leak tons of, of, of money there if you don't get it right.

Or you're just going to have somebody like continuously— it's like you're in Cloud Code where you're hitting the yes button and just they're just constantly hitting the buy more credits button. Yeah, I get what you're saying and that's fair. That's fair. I think some of the values of using custom agents are it's native to your collaboration tool. Whenever you're working in these other software, I like Cloud Code, which by the way, or Cloud Cowork and just Cloud Enterprise in general, it's absolutely incredible. It doesn't feel like you're working with other people and it very much feels like you're a person that is injecting your work in other places or doing it all on your own zone where Notion, it's a, it's a place for collaboration.

It's a place where, um, your entire company's workspace is, is living and existing.

And I think one, that gives you the feel of your, your AI agents are working with you and your team, um, in a different way. Uh, I think another one is being native to Notion gives you the ability, or like the custom agents being native in Notion gives you the ability to build inside the tool that you're working in as opposed to jumping into another zone. And I think there's a lot of value there, like custom agents.

Seamlessly connect to Slack, uh, and, and a Notion database to create really awesome workflows. Um, or it seamlessly connects to your knowledge database to create a chatbot with your knowledge database, or it seamlessly connects with your databases to like monitor and, and involve in those. Um, where Claude's working through an MCP to execute these actions, which is inherently more inefficient.

Um, so I think that that's one of the main reasons why people would, would, like why custom agents like drive or like is a value or has a value proposition associated with it.

Yeah. Yeah. And, and, uh, that's, that's exactly kind of what I wanted to hear. Like, keep me honest as well. What do you, would you push back that, um, I don't know. I, on the custom agent token usage and cost, and this is where it's a black box. It's like $10 for 1,000 tokens. I don't know what 1,000 tokens will get us. So remains to be seen. But push back on me. Do you think that, um, it actually is kind of a no-brainer to, to go the Notion token route?

I think that it is friction and it causes people to think a lot more and it, and it's hard, um, to, to really evaluate and quantify. Um, but I think if you take a very intentional approach to how you are creating agents, I think it's, it absolutely makes sense to do it or at least pursue it in a heavy way.

And I guess what I mean by that is you have the ability to open custom agents to your entire workspace, or you have the ability to open custom agents up to a handful of people. And if you take a really intentional, governed approach to custom agent creation, you will likely be able to drive a lot more value for the organization as opposed to giving every— giving— putting it in the power of everyone. You're going to end up with a lot of agents for one person as opposed to a few agents for everyone. So I think the value prop depends on how you govern and deploy it.

Yeah, you kind of already, uh, uh, snuck into the, my next question, which was what's our strategy for, how do you, you know, how do you effectively use agents, not just for yourself, but for, for a team? And I mean, we're, we're kind of touching in the same problem area that we solve with Notion in general, which is makes sense for one person or small team. You can, you can break stuff and it's fine. Your comms are pretty minimal. You just get on the same page. But for a large company, one, you could have a lot of agents that are stepping on each other's toes at minimum. But at worst, you have a lot of agents that are burning a lot of tokens that are really inefficient, that are doing a lot of the same things that are, you know, one's breaking something that the other one is fixing. You know, you have agents that are, I think you mentioned in another episode, using the same like Slack emojis to do different things. Like, that, that just sounds like a recipe for disaster. Right. So again, you know, it's like, we're not trying to, we're not trying to prescribe down to a T how people use Notion to the point that they don't get the flexibility of the tool anymore, but there really does need to be some sort of framework when you're trying to make it work for 1,000 people or, or even a couple hundred. Right. Yeah.

I think as, Well, there's so many things, but one is the operating system that your agents are working on.

That's an important component. And how do you— like, what are the permissions that you provide access to to make those work? Two is what are the different types of agents that you're going to create and allow in an organization? And then three, who's involved in like making the decisions and prioritizing and governing and maintaining the agents? And I think all three of those are really important when it comes to managing this for 100-person, 1,000-person, 10,000-person workspaces.

What is one agent that you think somebody could build for their company tomorrow and it will immediately be valuable, almost unconcerned with what kind of business they're in?

There are a lot of agents you could build tomorrow if your datasets are good.

That would be really valuable for any business. But I'm actually going to answer that in two parts. One is it's super easy to create a custom agent. You can vibe code a custom agent in just a couple of minutes. I think it's very— it's much harder to build a custom agent that is built really well and sturdy and predictable and scales with your organization. So while it's really easy to build them, I still think that we need to be encouraging frameworks for how you approach the creation of these agents, or else you're going to end up with a bunch of mediocre agents. So not answering your question, but just highlighting that while you can do it really quickly, you should still do it with intent. But to answer your question directly, it's if you have a solid knowledge hub, let's just use the people team.

Everyone actually focuses on building strong people team knowledge for an organization like your total rewards and your out-of-office policy. And how do I onboard to this company?

Like these sorts of items, like those are typically in one central place. Maybe they're a bunch of sticky pages in one page or one like people page, or maybe you have a people database or a blueprint. You typically have this one one dataset that has all of this people information. You also typically have a Slack channel that is Ask People. And while you have your knowledge database here, you have your conversations in Slack in another location, and people are asking, what's our out-of-office policy in Slack? And then some human is writing back, here's a link to our out-of-office policy. So an agent that you could build tomorrow, if this is your situation, is an agent that's trained specifically on that people database that's connected to that Ask People Slack channel that's responsible for responding responding to any questions that people ask about that database. And if it has high confidence, it answers. If it doesn't, it tags the PeopleOps person responsible and creates a ticket for them to go fix that resource or add a resource to it. So that, that's an example of one that just about any organization could probably stand up tomorrow and save a good amount of time for people.

Yeah. Let me play devil's advocate here, which I know that we love these kinds of terms. Why is that better than asking Nosy?

It's probably not just the way of working that exists today.

Slack, you mean like people just default to the path of least resistance?

People kind of default to have to asking a human to answer the question for them. And that will change. But it's certainly the fact of today.

It's like I would rather say, Josh, what is this?

As opposed to ask Nosy, Nosy, what is this? And if I feel like I'm asking someone in a channel, it doesn't matter who responds if the right answer is what is responding.

Even if it's a bot.

Yeah.

You know, you've got other human eyes on it. Yeah. Yeah.

Because something that happens in those channels that's really beneficial too is you start to get like when the answers are correct, you have multiple people seeing that and giving positive emojis to it and you actually start to see celebration around that agent, which encourages trust and development in it.

Yeah, I'm, I'm noticing Trying to boil, you know, Notion away and Slack away, but like whatever you'd call these platforms, they're so interlocked, Slack and Notion, like comms and knowledge, you know, or artifact collections, right? If I'm gonna get really just like down to what it is, which makes it surprising that We haven't seen like a Notion replacement of Slack. Do you think that that's a good separation of concerns? Or do you see a world where these are, do you see these like interlocked just fundamentally?

I actually think that you're highlighting something that might feel like it's novel to our generation and where we are with technology, but it's, it's the way business has always been. Um, the, before we had Slack and Notion, we had email and Microsoft Word, right? Um, and that eventually turned into Microsoft 365 and email. But nonetheless, it's like we've always, like, connecting comms and notifications to work that's happening is something that has, has always been happening. It's just right now it's generally speaking Slack and Notion, or it is Notion and Google Meet, um, and, and, and, and Gmail, or it is Office 365, which is a combination of all of these, right? So like, I think that that's what you're having, but it's very much comms and notifications and operational work that's taking place.

Yeah. Yeah. Thank you for asking my question for me and answering it.

But it's an important highlight here is custom agents are really good at filling that gap. And the gap here is transferring knowledge. And that's something that you hire people for. You have someone that's job is responsible for doing status updates. Or for planning meeting agendas or for creating your quarterly business reviews. Like people's jobs are those things, but that's just aggregation of knowledge across different platforms or across different people or different teams. It's just the aggregation of knowledge. And custom agents really, really easily fill that gap because you can say on this trigger, look at the last week of meetings, look at the last week of actions that are incomplete, create a status update and send that to this Slack channel. Like that's a very easy thing that you can do now. And it's all about What are those gaps that, that, that people are doing that they don't like to do and creating a, a solution for it?

Yeah. Yeah. Well said. Makes me think there's this, this kind of sentiment, uh, on the internet right now, or has been for a little bit, that what is, what is UI? What will UI be? Will it exist? Will we just interface with software or information just via chat? Um, now that now the interfaces can kind of be created almost at will now. Like they'll get faster and better, you know, in no time. And so it kind of begs the question, like, what is Notion? And there's, there's even an idea that like Notion stays your backend while you build things on top. Is, I never thought about this, but like, is Slack the way, is Slack gonna be that, that interaction layer or that chat layer? I don't know. Probably not. But. How do we, how do we think about this? How do we layer on top of, of our info, of our data? We're like, we're connecting comms to, to information. Custom agents are filling that gap, but like, where, where do we go from here in the next few years?

I think the important thing to highlight here is people use Slack because it's very easy to communicate on, right? You can jump to a channel and ask a question, or you can raise an alert, and it's very easy to communicate on, but all that occurs on Slack is the comms. No knowledge is being made— is being made there and stored there. No firm decisions. Well, they're probably being made there. They're not being recorded there. The action items that are being sent there, they're not being created in Slack. So Slack is just the comms layer. And maybe that's actually the point you're getting to. But the point that I'm getting to is that's actually really inefficient. The really inefficient place is where your knowledge and operations are taking place. That's where your comms should be taking place and where your collaboration interaction should be taking place. And if you're doing that really Maybe I'm actually wrong that that's where it should be taking place, but that's where all of the information should be stored. And then it's the AI layers on top of it that are probably actually doing the coordination, collaboration, and the commenting and the notification management. And what you need, the user needs, is a way to surface out what is relevant to them or be able to find what is, what is relevant to them in an easier way. And that's, I think, what years from— or like in the near future, that's like the abstraction layer that's going to exist and that Notion will, will really help be helpful in creating.

Yeah, I think, I think what is kind of being highlighted here for me is the necessity to cater to human inefficiency. Uh, and that's, that's not a bad thing, right? Like humans are the reason we're doing any of this. We're not doing this to serve the bots. The bots, you know, we use bots to, to help serve the goals that we have. But humans just kind of have funny quirks. We don't do things the most efficient, but yet we have something different that's deep. I don't know, like deep in, as part of our humanity, we're, you know, we're like maybe what I would call like truly creative. Wait, that's for a different discussion anyways. But there's like a necessity to kind of support the ways that humans just work even if it's not the most efficient. And that's a real part of, that's a real part of work.

Absolutely. I think it actually aligns quite well with the framework that we've been running with, with the clients lately is what are the types of custom agents that exist? And it's every place where there's an inefficient layer involved in the workflow. One is your knowledge and Q&A agents. They're responsible for being able to analyze large sets of data and provide an accurate response for it. Where historically it was that person asking someone else What is the answer to that? There are the content creation agents where every quarter we need to create a quarterly business review for our client, or every week we need to create a status update, or every account we need to create an account plan. These again are just aggregation of large amounts of information, and a human would typically take weeks on an account plan and weeks on a quarterly business review. And if you're able to connect your Gong transcripts and your Slack messages and your Notion operational information all together, these things start to be created on a much easier cycle. The third one is monitoring and alerting. It's like, what do I— what's relevant to me that I don't know? And these are things where typically it's, Leslie, can you run through all of our things and tell me, like, what are the important things for me to look at? But ideally, that's not a human that's responsible for doing that. That is somebody that is an agent that's reviewing all of these components. And then the fourth area here is this, is this, uh, Notion creates data, data analysts out of everyone, uh, where like they have to go manual, like for your data to be good, a human needs to go in there and add in all of the metadata correctly and they need to do that every single time, but they shouldn't have to. And like, that's another area where agents can, can, can release that, release that, uh, that, that inefficiency. So I guess like when we're, when we're thinking about all these inefficiency gaps and where custom agents can really fit in. I think it falls into those 4 categories pretty, pretty cleanly.

Yeah. Yeah. When, when Notion, uh, released or publicly kind of released Notion 3.0, right? It was all about obviously AI, but it was about like, it was about busy work. It was like getting rid of the busy work, um, so that you can kind of return to the work that you really are talented at, that only you can do. And I think that's been kind of our north star. At least in my head for a long time, is like, how do we like remove the meta work, the busy work, whatever you want to call it, for people to actually just be good at what they do, to let people cook, you know, so to speak. And that was actually my, my, my like personal mission statement for a little bit, let people cook. That applies to our business as well as like our clients. That's what we all want at the end of the day. And that is the vision for what agents can help people in their day-to-day work. And I think that's a reality for the next couple years at least. We'll see what happens in 2 years, but for now, the future is at least, you know, there is a light at the end of the tunnel. It looks bright instead of dark. I don't know.

Hopefully.

Yeah.

I think there are some scary futures that we could potentially go down to, but we'll see.

Yeah.

Yeah.

All right.

Well, as we wrap up, maybe we'll leave people with something that they can take away and actually use if they don't want to file away all of our ramblings and philosophy. There's a lot of— there's a lot of gold in there if you dig. But how can— as people start to use custom agents and build them, especially considering that there's going to be, you know, real cost, but they want to drive real value out of it. What is like a, what are some best practices for actually how to set up a custom agent well so that you can start at, you know, 50, 60% rather than 0 to 10%?

Something that I've, I've become really intentional about when I'm teaching people how to create these or when we're creating them is actually approaching the creation of a custom agent with a framework. It's, and this will help you get to the result that you want. Like, first component is what is it that you're trying to do? What is the desired outcome and what should it look like? What do you want this custom agent to do? To accomplish? Is it a Slack message that if answered correctly, it ends it in Slack, and if answered incorrectly, it creates a ticket in Notion? Understand what is your outcome that you want to do. And then you go to the front of this and it's understand what are the data inputs that are going to inform the output of this agent. If going back to that people one, it's like, is it the people database? Is it a people database and IT? Is it a private people database plus your, your company intranet zone? What are the core databases and data inputs that are going to be here? And being very intentional there will, one, protect you from not opening up incorrect sources, but two, it'll help you be really thoughtful.

Like, where do you create things moving forward?

Because if you find yourself linking to 10 individual pages, your architecture isn't very good and it's not super scalable.

So it's understanding like, what are the data sources that are going to inform that?

And then once you're able to have those two, you have like the front end and the back end of this pulled together.

That's when you start to work with the solution to, to, to, to work with the custom agent to design what are the, the pipes that need to be built between these two to, to create your output.

And I think that that's, that's the way that I'm thinking about them right now.

Yeah. Yeah. Honestly ties really well into something that I think is just fundamental to, to work, whether AI is involved or not. It, it was true 10, 500, 1,000 years ago. It's all about inputs and outputs, right? When you boil down work, it's about inputs and outputs. What do you do when you receive the inputs and you know what's being done to it before it gets output to the next thing? And as work gets more complex, there's a complex network and web of where, you know, especially now that we're dealing with knowledge and knowledge work and not like, you know, physical manufacturing or things like that. There's this whole, you know, nebulous, what feels like a nebula of where the thing could go and who needs to pick it up and grab it from where and do something with it. But at the end of the day, if you look at work as inputs and outputs and almost like a factory. That's where you install custom agents and that's how you think about custom agents. Think about what do you need out of it and what does it have available to it? And then what are you going to ask it to do to add value, to go full circle? If it's value creation, that's what it's all about is augmenting the input and adding value to it before you pass it off.

Yeah. Couldn't have said it better.

All right. That's it for Notion State of Mind podcast this week. We'll, uh, we'll keep iterating on this, especially as agents and AI develops, but I promise we won't become an AI podcast, but we'll still try to stay in our lane. Uh, but until then, appreciate any feedback, comments, love, hate, that even helps our algorithm. So if you, if you hate this, give us one star and, and, uh, thank you and thank you very much and we'll see you in the next one.